Onediff Support #183
Open
Onediff Support #183
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…ors (livepeer#66) This commit ensures that batches in the T2I pipeline are processed sequentially. This change is necessary because we currently lack the ability to estimate a GPU's VRAM capacity and manage requests accordingly. * process text-to-image requested image count sequentially * refactor: cleanup sequential images code This commit cleans up the sequential images code a bit. --------- Co-authored-by: Rick Staa <[email protected]>
This commit ensures that a more descriptive error is thrown when the huggingface token is not set.
…er#91) * pin setuptools to 69.5.1, breaks build if 70.0.0 is used * pin packages to current version
* add saftey checker to image to video * refactor(runner): apply black formatter This commit applies the black formatter onto the codebase. --------- Co-authored-by: Rick Staa <[email protected]>
This commit ensure that the NSFW flag is set as optional. This was done since a None list can be returned.
This commit removes the NSFW log statement in the I2V pipelines and reorders the argument parsing in the T2I pipeline.
* feat(runner): disable DEEPCACHE for lightning/turbo models This commit ensures that people can not use the deepcache optimization with lightning and turbo models. As explained in livepeer#82 (comment) this optimization does not offer any speedup for these models while it does reduce image quality. * fix: ensure correct lighthing/turbo matching This commit ensures that Lightning/Turbo model names with both the `-` and `_` are matched when checking whether the orchestrator loaded a Lightning/Turbo model.
This commit updates the oapi-codegen tool to the latest version.
* Add support for the `timbrooks/instruct-pix2pix` model * Remove text-to-vid model * Tweak pix2pix * Do not randomize guidance scale * Make image guidance scale a param * Improve the image-to-image processing by enabling the timbrooks/instruct-pix2pix model. * merge all stronk changes for image-to-image improvements * minor update * regenerated runner.gen.go * refactor(I2I): improve pix2pix pipeline initialization This commit removes arguments that are not used during the initialization of the pix2pix pipeline. It also cleansup the codebase a bit and changes the default to improve transparency. * chore: update openapi spec This commit updates the OpenAPI spec and removes the openapi.yaml file since it is not directly used. --------- Co-authored-by: Marco van Dijk <[email protected]> Co-authored-by: Rick Staa <[email protected]>
This commit ensure that the warning statement that has been printed to the console since oapi-codegen v2.2.0 does not cause a syntax error.
This commit pins oapi-codegen to prevent the golang bindings from breaking due to upstream changes.
This commit removes the codegen install command from the makefile. This is no longer needed now that go run accepts specific version postfixes (see https://go.dev/doc/go1.17).
…est (livepeer#95) * update img2img to do sequential processing of batch request * refactor(runner): improve consistency between I2I and T2I pipelines This commit enhances the consistency between the I2I and T2I pipelines, making them easier to compare. --------- Co-authored-by: Rick Staa <[email protected]>
This commit cleans up the download script to remove redundancy.
…ivepeer#98) This commit ensure that the multipart writers correctly forward the `safety_check` and `image_guidance_scale` parameters to the AI runner container.
This commit applies some small code and documentation formatting improvements.
* add upscale image support using stabilityai/stable-diffusion-x4-upscaler model * Add host port for upscale pipeline * added model download to the dl_checkpoints.sh * fix: fix some small bugs and improve formatting This commit fixes some small bugs and improves the code formatting so that it is more in line with the other pipelines. * fix: ensure upscaling OpenAPI spec gets created This commit ensures that the `gen_openapi.py` file also creates the OpenAPI spec for the upscaling route. It also updates the Golang client bindings. * fix(worker): fix incorrect automatic client types This commit ensures that the right client request and response types are used. --------- Co-authored-by: Elite Encoder <[email protected]> Co-authored-by: Rick Staa <[email protected]>
This commit ensures that the download script downloads the right alpha models when the `alpha` flag is specified.
This commit ensures that the right format is set in the OpenAPI spec for when the I2I response is coming from the gateway. This is a little bit hacky since we don't yet generate the OpenAPI spec where it is implemented (i.e. in go-livepeer).
This commit ensures that when people click the feature request issue button they are redirected to our roadmap and feedback system.
This commit replaces the model and pipeline issue templates with a link to the roadmap and our feature request system.
This commit fixes a small typo in the Feature Request issue button.
This commit adds the prerequisites to the vscode dev container instructions.
This commit makes some very small code improvements that make applying a mock patch easier.
This commit ensures that numpy<2.0.0 is used to prevent conflicts with Pytorch 2.1.1. We can upgrade these packages in the future.
This commit improves the I2I code formatting.
This commit adds all the components needed to be able to mock the pipelines.
Replaces the old mock.patch file with a new one that ensures people can also use ENVIRONMENT variables directly. See livepeer#116.
This commit ensures that developers have all the steps needed to debug a ai-runner container when it is used in the go-livepeer library.
This commit introduces support for the Stable Diffusion 3 Medium model from Hugging Face: [https://huggingface.co/stabilityai/stable-diffusion-3-medium](https://huggingface.co/stabilityai/stable-diffusion-3-medium). Please be aware that this model has restrictive licensing at the time of writing and is not yet advised for public use. Ensure you read and understand the [licensing terms](https://huggingface.co/stabilityai/stable-diffusion-3-medium/blob/main/LICENSE) before enabling this model on your orchestrator.
Add audio to text pipeline --------- Co-authored-by: Rick Staa
This commit applies several code formatting improvements and replaces the hardcoded error codes by the error codes in the FastAPI status module.
…vepeer#123) * Update image_to_image.py added variable num_inference_steps and added the same variable in pipeline function calling with default value of 50 * Update openapi.json added property for image to image of number of inference steps. * Update for num_inference_steps * Update image_to_video.py * Update image_to_image.py * Update image_to_video.py * Update openapi.json * Update image_to_video.py * Update image_to_image.py * Update image_to_video.py * Update openapi.json * feat: add `num_inference_steps` to upscale pipeline This commit adds the `num_inference_steps` parameter to the upscale pipeline. * feat: improve I2I pipeline num_inference_steps behavoir This commit ensures that the `strength` is set correctly when the sdxl-turbo model is used and cleans up the `num_inference_steps` implementation. * feat: improve I2V num_inference_steps behavoir This commit improves the I2V `num_inference_steps` defaults and code. * feat: improve T2I num_inference_steps behavoir This commit improves the implementation and defaults for the num_inference_steps parameter. * feat(worker): add num_inference_steps to I2I, I2V and Upscale pipelines This commit adds the `num_inference_steps` to the I2I, I2V and Upscale pipelines in the worker codebase. --------- Co-authored-by: Jason <[email protected]> Co-authored-by: Jason Stone <[email protected]>
* enable multiple prompts for T2I * refactor(runner): make prompt splitting more general This commit ensures that the prompt splitting logic implemented in the previous commits can be reused in multiple pipelines. --------- Co-authored-by: Rick Staa <[email protected]>
This commit adds a MIT license to the codebase.
This commit adds a simple GitHub action that can be used to trigger a OpenAPI spec update in the https://github.com/livepeer/docs repository.
This commit replaces the hardcoded repository dispatch with the https://github.com/peter-evans/repository-dispatch action.
This commit improves the OpenAPI upstream sync action syntax.
This commit adds the trigger commit to the repository-dispatch call.
This commit adds the SDK workflow trigger to the OpenAPI sync github action.
This commit improve the OpenAPI sync action syntax.
* ci: add PR OpenAPI/bindings check action This commit adds a simple action that notifies developers when they forgot to update the OpenAPI spec or golang bindings. * ci: improve OpenAPI/bindings change action This commit improves the code of the OpenAPI and golang bindings check action.
This commit ensures that the https://huggingface.co/SG161222/Realistic_Vision_V6.0_B1_noVAE model is supported in the T2I pipeline.
This commit adds a update trigger to the OpenAPI sync action that triggers a update of the JS/TS SDK.
This commit addes a trigger to update the OpenAPI spec in https://github.com/livepeer/ai-sdk-js. Furhter it improves the OpenAPI spec upstream sync action to forward more information.
This commit adds parameter annotations to the T2I pipeline similar to how it is done in the rest of the pipelines. Descriptions will be added in a subsequenty commit.
This commit sorts the python imports using the https://pypi.org/project/isort package.
This commit ensures the right upstream repos are triggered in the trigger upstream OpenAPI sync action.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
As for What i did i added an onediff optimization support under which i made some changes in the following files
Issue Posted Upon
#livepeer/bounties#37
Previous Pr
#149
File Changes
DockerFile - Added onediff and now repo works on torch 2.3.0 with other libs on
pipelines/text-to-image.py
routes/text-to-image.py
bench.py - make it optimized for adding multiple optimization
How to run this
Download all the model as mention in readme.md under runner dir
Build the docker image
docker build -t ai-runner-benchmark .
Run the bench.py
docker run --gpus all -v ./models:/models ai-runner-benchmark:latest python bench.py --pipeline text-to-image --model_id SG161222/RealVisXL_V4.0_Lightning --runs 3 --batch_size 1 --num_inference_steps 10 --optimization onediff