For ChatGPT if we set the maximum number of tokens (say 100k) beyond the 4,096 will it adhere to that? Or is there a hard limit of 4,096 in active pieces that can’t be overridden?
Most ChatGPT models support token limits much greater than the 4,096.
“The maximum number of tokens to generate. Requests can use up to 2,048 or 4,096 tokens shared between prompt and completion depending on the model. Don’t set the value to maximum and leave some tokens for the input. (One token is roughly 4 characters for normal English text)”
