Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added better censored check per-image when submitting job #298

Closed
wants to merge 2 commits into from

Conversation

rbrtcs1
Copy link
Contributor

@rbrtcs1 rbrtcs1 commented Sep 29, 2024

I just quickly typed this up on my phone, so it’s definitely worth double checking for correctness.

But I think this should fix the “censored” value inside each generation object to actually be whether that image was censored instead of the current situation which is if any image in the batch was censored.

@rbrtcs1
Copy link
Contributor Author

rbrtcs1 commented Sep 29, 2024

Well it seems like the gen metadata at this stage is the model from the sdk rather than a dict. Not sure what the correct way is to access the ‘type’ attribute for comparison (as a string). But it seems that that is the issue failing the tests here.

@db0
Copy link
Member

db0 commented Sep 29, 2024

The censored payload from the worker is obsolete and discarded on the horde in favour of parsing gen_metadata as can be seen here https://github.com/Haidra-Org/AI-Horde/blob/b520fe3c2c998e7141dfe1e111003834448de186/horde/classes/stable/processing_generation.py#L91

This was an old boolean for the whole job, before we were doing batches, so it's by nature inaccurate.

@rbrtcs1
Copy link
Contributor Author

rbrtcs1 commented Sep 29, 2024

It’s possible I’m misunderstanding something here, but my understanding is as follows:

  1. the worker sets the censored value equal to true if any image in the batch was censored. It is false only if all images in the batch are ok.
  2. the horde server receives that value, then overrides to it to true if the submitted job has a censored gen_metadata.

This seems to me that when 1 image in a batch was censored but the others were not, they would still all be marked as censored. Unless I’m misunderstanding the aihorde code and the censored value defaults to False. (Each good image in that batch would start as censored=true, then the horde server wouldn’t override them, leaving them set to true)

@db0
Copy link
Member

db0 commented Sep 29, 2024

Look at this change from today Haidra-Org/AI-Horde@8fab6e4

@rbrtcs1
Copy link
Contributor Author

rbrtcs1 commented Sep 29, 2024

I must have missed that, looks good!

@rbrtcs1 rbrtcs1 closed this Sep 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants