Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

vertexai: seed missing in GenerationConfig #11265

Open
erikdubbelboer opened this issue Dec 11, 2024 · 2 comments
Open

vertexai: seed missing in GenerationConfig #11265

erikdubbelboer opened this issue Dec 11, 2024 · 2 comments
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.

Comments

@erikdubbelboer
Copy link

According to these documentations ([1], [2]) seed is supported by the API, I guess the generated code just needs to be updated to support this.

Missing here:

// GenerationConfig is generation config.
type GenerationConfig struct {
// Optional. Controls the randomness of predictions.
Temperature *float32
// Optional. If specified, nucleus sampling will be used.
TopP *float32
// Optional. If specified, top-k sampling will be used.
TopK *int32
// Optional. Number of candidates to generate.
CandidateCount *int32
// Optional. The maximum number of output tokens to generate per message.
MaxOutputTokens *int32
// Optional. Stop sequences.
StopSequences []string
// Optional. Positive penalties.
PresencePenalty *float32
// Optional. Frequency penalties.
FrequencyPenalty *float32
// Optional. Output response mimetype of the generated candidate text.
// Supported mimetype:
// - `text/plain`: (default) Text output.
// - `application/json`: JSON response in the candidates.
// The model needs to be prompted to output the appropriate response type,
// otherwise the behavior is undefined.
// This is a preview feature.
ResponseMIMEType string
// Optional. The `Schema` object allows the definition of input and output
// data types. These types can be objects, but also primitives and arrays.
// Represents a select subset of an [OpenAPI 3.0 schema
// object](https://spec.openapis.org/oas/v3.0.3#schema).
// If set, a compatible response_mime_type must also be set.
// Compatible mimetypes:
// `application/json`: Schema for JSON response.
ResponseSchema *Schema
}

1: https://cloud.google.com/vertex-ai/docs/reference/rest/v1/GenerationConfig
2: https://cloud.google.com/vertex-ai/generative-ai/docs/model-reference/inference#request

@erikdubbelboer erikdubbelboer added the triage me I really want to be triaged. label Dec 11, 2024
@codyoss codyoss added type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design. and removed triage me I really want to be triaged. labels Dec 11, 2024
@codyoss
Copy link
Member

codyoss commented Dec 11, 2024

cc @jba @eliben

@eliben
Copy link
Contributor

eliben commented Dec 12, 2024

cc @happy-qiao

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: feature request ‘Nice-to-have’ improvement, new feature or different behavior or design.
Projects
None yet
Development

No branches or pull requests

3 participants