Skip to content
    Open navbar menu
    Sign UpLog In
    Back to all Bounties

    Earn 363,600 ($3,636.00)

    Time Remainingdue 1 year ago
    Completed

    Playground for local models

    natfriedman
    natfriedman
    Posted 1 year ago

    Accepted Application

  1. 0xhuda
    0xhuda
    Accepted
    Applied 1 year ago
    Remote inference demo: https://playground.0xhuda.repl.co/ Tweet with local inference demo: https://twitter.com/iamzainhuda/status/1619058598352674817?s=20 Hi Nat! I've created a demo for you to try out (remote inferencing only) and also I've included a tweet where I go over local inferencing as well (loading hugging face models locally) and it extends to any text-generation model that is available on the platform (works with any T5 finetune you upload there) I built it with flask and react with all logic, inference, and such lies on the server side to make it easy to change as needed and as well as extend to future providers or other ways to load models locally. The front end also dynamically displays parameters available for different models (cohere generation params and ranges differ from huggingface ones). The following features are implemented in the demo: + Inferencing of remote models (Cohere is chosen as the provider, can be extended easily to other providers) + Model selection + Generation parameter selection + Undo/Regenerate buttons + Save/Load presets + T5 models locally (view the twitter link for a demonstration of this one) I don't have much other public work but at facebook I launched a all-in-one performance profiling service that collects metrics, traces, and computes GPU analyses on over 8000 training jobs a day on Facebooks training cluster. Used by over 300 MAU. https://pytorch.org/blog/performance-debugging-of-production-pytorch-models-at-meta/ Thanks for your time and consideration!
    Twitter
    GitHub
  2. Applications

    Copyright © 2024 Replit, Inc. All rights reserved.
    • twitter
    • tiktok
    • instagram
    • facebook

    Replit

    Programming languages

    • Python
    • JavaScript
    • TypeScript
    • Node.js
    • Nix
    • HTML, CSS, JS
    • C++
    • Golang