Run models on sensible generation defaults out of the box
A fresh install had no generation settings until someone set one, so every model ran on whatever its provider happened to use. Each harness now has a default (text 0.3 temperature, vision 0.2, image the provider's own), shown in the model settings modal so what a model runs on is never a guess. The defaults are only sent to a model whose catalog lists the field, so a provider that says nothing about its models still gets no parameter it might reject; anything saved on a harness or bot wins over the default.
This commit is contained in:
@@ -1,5 +1,7 @@
|
||||
import {
|
||||
HarnessParameters,
|
||||
harnessParameterValue,
|
||||
modelAdvertisesParameter,
|
||||
modelTakesParameter,
|
||||
} from "@/src/dict/harness-parameter-dict";
|
||||
import grabHarnessSetting from "@/src/functions/backend/db/harness/grab-harness-setting";
|
||||
@@ -13,10 +15,10 @@ type Params = {
|
||||
|
||||
/**
|
||||
* A model's generation settings for one call: what is saved in the
|
||||
* harness's setting (the workspace's, the bot's, else the default),
|
||||
* keyed by the field sent on the wire, keeping only what the model says it
|
||||
* takes. Empty when none are set, so the model gets its provider's
|
||||
* defaults.
|
||||
* harness's setting (the workspace's, the bot's, else the default), else
|
||||
* `HarnessParameterDefaults` so a fresh install is ready to go. Keyed by
|
||||
* the field sent on the wire, keeping only what the model says it takes
|
||||
* (a default only where the model explicitly advertises it).
|
||||
*/
|
||||
export default async function grabChatParams({
|
||||
harness,
|
||||
@@ -34,7 +36,11 @@ export default async function grabChatParams({
|
||||
return (model) =>
|
||||
Object.fromEntries(
|
||||
HarnessParameters.flatMap((parameter) => {
|
||||
const value = setting.options[parameter.key];
|
||||
const { value, saved } = harnessParameterValue({
|
||||
harness,
|
||||
params: setting.options,
|
||||
key: parameter.key,
|
||||
});
|
||||
|
||||
if (typeof value != "number" && typeof value != "string") return [];
|
||||
if (!value && value !== 0) return [];
|
||||
@@ -45,6 +51,14 @@ export default async function grabChatParams({
|
||||
})
|
||||
)
|
||||
return [];
|
||||
if (
|
||||
!saved &&
|
||||
!modelAdvertisesParameter({
|
||||
parameter,
|
||||
supported: model.supported_parameters,
|
||||
})
|
||||
)
|
||||
return [];
|
||||
|
||||
const key = "wire" in parameter ? parameter.wire : parameter.key;
|
||||
|
||||
|
||||
Reference in New Issue
Block a user