No obvious way to choose generation settings

[rcchan123]

Right now, I can't figure out how to alter generation settings, such as temperature, rept-penalty, tail-free sampling etc, and I also don't know how to change maximum generation length. Digging around, it seems that the option used to be in cogs/pygbot.py, but it doesn't seem to be in there now.

[ausboss]

you are right it is not obvious. I have been wanting to add a feature that makes it easy.

Here is a workaround if you are not afraid of editing a line in python, you can go to this line in the discordbot.py and add custom parameters after the endpoint.

Example
before:

bot.llm = KoboldApiLLM(endpoint=bot.endpoint)

after:

bot.llm = KoboldApiLLM(endpoint=bot.endpoint, temperature=0.75, max_length=512)

Here I added temperature and max length parameters. Don't forget the commas.

A list of all the parameters can be seen here:
https://api.python.langchain.com/en/latest/llms/langchain.llms.koboldai.KoboldApiLLM.html

[rcchan123]

Ok, Just want to say that this works perfectly, thanks!
I will still leave this issue open though, as I think we both agree it it neither obvious nor intuitive.