> ## Documentation Index
> Fetch the complete documentation index at: https://docs.siliconstorm.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Request Parameters

<table cellpadding="4" cellspacing="0" frame="border" border="1" rules="all" data-header="7">
  <thead align="left">
    <tr>
      <th align="left" colspan="4" valign="top"><p>Parameter</p></th>
      <th align="left" valign="top"><p>Required</p></th>
      <th align="left" valign="top"><p>Description</p></th>
      <th align="left" valign="top"><p>Value Requirements</p></th>
    </tr>
  </thead>

  <tbody>
    <tr>
      <td colspan="4"><p>model</p></td>
      <td><p>Required</p></td>
      <td><p>Model Name</p></td>
      <td><p>Must match the value of `modelName` in the MindIE Server configuration file.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>messages</p></td>
      <td><p>Required</p></td>
      <td><p>Inference request message structure.</p></td>
      <td><p>List type, with the number of characters in the `messages` content being between 0KB and 4MB. Supports both Chinese and English. The number of tokens after tokenization should be less than or equal to the minimum value among `maxInputTokenLen`, `maxSeqLen - 1`, `max_position_embeddings`, and 1MB. The `max_position_embeddings` is obtained from the weight file `config.json`, and other related parameters are taken from the configuration file.</p></td>
    </tr>

    <tr>
      <td rowspan="12"><p>-</p></td>
      <td colspan="3"><p>role</p></td>
      <td><p>Required</p></td>
      <td><p>Inference request message role.</p></td>

      <td>
        <p>String type. Possible roles are:</p>

        <ul>
          <li>system: System role</li>
          <li>user: User role</li>
          <li>assistant: Assistant role</li>
          <li>tool: Tool role</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="3"><p>content</p></td>
      <td><p>必选</p></td>
      <td><p>推理请求内容。单模态文本模型为string类型，多模态模型为list类型。</p></td>

      <td>
        <ul>
          <li>
            <p>string:</p>

            <ul>
              <li>If the role is 'assistant' and tool\_calls is not empty, content can be omitted; for other roles, content must be provided.</li>
              <li>In other cases, content must be provided.</li>
            </ul>
          </li>

          <li>list: Follows the example format of multimodal model 'inputs' parameter.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td rowspan="5"><p>-</p></td>
      <td colspan="2"><p>type</p></td>
      <td><p>Optional</p></td>
      <td><p>Type of inference request content.</p></td>

      <td>
        <ul>
          <li>text: Text</li>
          <li>image\_url: Image</li>
          <li>video\_url: Video</li>
          <li>audio\_url: Audio</li>
        </ul>

        <p>The total number of image\_url, video\_url, and audio\_url in a single request should be \<= 20.</p>
      </td>
    </tr>

    <tr>
      <td colspan="2"><p>text</p></td>
      <td><p>Optional</p></td>
      <td><p>The inference request content is text.</p></td>
      <td><p>Cannot be empty, supports both Chinese and English.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>image\_url</p></td>
      <td><p>Optional</p></td>
      <td><p>The inference request content is an image.</p></td>
      <td><p>Supports images from server local paths, image types support jpg, png, jpeg, and base64 encoded jpg images. Supports image URLs with both HTTP and HTTPS protocols. The maximum image size supported is 20MB.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>video\_url</p></td>
      <td><p>Optional</p></td>
      <td><p>The inference request content is a video.</p></td>
      <td><p>Supports videos from server local paths, video types support MP4, AVI, WMV, and video URLs with both HTTP and HTTPS protocols. The maximum video size supported is 512MB.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>audio\_url</p></td>
      <td><p>Optional</p></td>
      <td><p>The inference request content is audio.</p></td>
      <td><p>Supports audio from server local paths, audio types support MP3, WAV, FLAC, and audio URLs with both HTTP and HTTPS protocols. The maximum audio size supported is 20MB.</p></td>
    </tr>

    <tr>
      <td colspan="3"><p>tool\_calls</p></td>
      <td><p>Optional</p></td>
      <td><p>Tool calls generated by the model.</p></td>
      <td><p>Type is List\[dict]. When the role is assistant, it represents the model's call to the tool.</p></td>
    </tr>

    <tr>
      <td rowspan="3"><p>-</p></td>
      <td colspan="2"><p>function</p></td>
      <td><p>Required</p></td>
      <td><p>Represents the tool invoked by the model.</p></td>

      <td>
        <p>Type is dict.</p>

        <ul>
          <li>arguments, required, a JSON-formatted string representing the parameters for the function call.</li>
          <li>name, required, a string representing the name of the function being called.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="2"><p>id</p></td>
      <td><p>Required</p></td>
      <td><p>Represents the ID of a specific tool invocation by the model.</p></td>
      <td><p>String.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>type</p></td>
      <td><p>Required</p></td>
      <td><p>The type of tool being invoked.</p></td>
      <td><p>String, only supports "function".</p></td>
    </tr>

    <tr>
      <td colspan="3"><p>tool\_call\_id</p></td>
      <td><p>Required when the role is "tool", otherwise optional.</p></td>
      <td><p>Associates with the ID of the model’s tool invocation.</p></td>
      <td><p>String.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>stream</p></td>
      <td><p>Optional</p></td>
      <td><p>Specifies whether the result should be text-based inference or stream-based inference.</p></td>

      <td>
        <p>Boolean type, default value is false.</p>

        <ul>
          <li>true: Stream-based inference.</li>
          <li>false: Text-based inference.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>presence\_penalty</p></td>
      <td><p>Optional</p></td>
      <td><p>The presence penalty ranges from -2.0 to 2.0 and affects how the model penalizes new tokens based on whether they have appeared in the text so far. Positive values will penalize words already used, increasing the likelihood of the model introducing new topics.</p></td>
      <td><p>Float type, value range \[-2.0, 2.0], default value 0.0.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>frequency\_penalty</p></td>
      <td><p>Optional</p></td>
      <td><p>The frequency penalty ranges from -2.0 to 2.0 and influences how the model penalizes new words based on the existing frequency of words in the text. Positive values will penalize frequently used words, reducing the likelihood of repetition in a line.</p></td>
      <td><p>Float type, value range \[-2.0, 2.0], default value 0.0.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>repetition\_penalty</p></td>
      <td><p>Optional</p></td>
      <td><p>The repetition penalty is a technique used to reduce the probability of repeating segments in text generation. It penalizes previously generated text, making the model more likely to choose new, non-repetitive content.</p></td>
      <td><p>Float type, value range (0.0, 2.0], default value 1.0.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>temperature</p></td>
      <td><p>Optional</p></td>
      <td><p>Controls the randomness of the generated output, with higher values producing more diverse outputs.</p></td>

      <td>
        <p>Float type, value range \[0.0, 2.0], default value 1.0.</p>
        <p>The higher the value, the greater the randomness of the result. It is recommended to use a value greater than or equal to 0.001, as values below 0.001 may lead to poor text quality.</p>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>top\_p</p></td>
      <td><p>Optional</p></td>
      <td><p>Controls the range of words considered by the model during generation by using cumulative probability to select candidate words until the cumulative probability exceeds the given threshold. This parameter can also control the diversity of the generated result by selecting candidate words based on cumulative probability.</p></td>
      <td><p>Float type, value range (0.0, 1.0], default value 1.0.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>top\_k</p></td>
      <td><p>Optional</p></td>
      <td><p>Controls the range of words considered by the model during generation by selecting only from the top k candidate words with the highest probabilities.</p></td>

      <td>
        <p>Int32 type, value range \[0, 2147483647], default value determined by the backend model when the field is not set. For more details, refer to <a href="mindie_service0062.html" data-outer="inner" data-href="mindie_service0062.html" data-doc="true" target="_blank" data-multiple-screen="true">documentation</a>.</p>
        <p>If the value is greater than or equal to vocabSize, the default value will be vocabSize.</p>
        <p>vocabSize is read from the config.json file under the modelWeightPath directory. It is recommended that users add the vocab\_size or padded\_vocab\_size parameters to config.json to avoid inference failures.</p>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>seed</p></td>
      <td><p>Optional</p></td>
      <td><p>Used to specify the random seed for the inference process. The same seed value ensures reproducibility of inference results, while different seed values increase the randomness of the results.</p></td>

      <td>
        <p>UInt64 type, value range \[0, 18446744073709551615]. If not provided, the system generates a random seed value.</p>
        <p>A WARNING may appear when the seed approaches the maximum value, but it will not affect usage. To remove the WARNING, you can reduce the seed value.</p>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>stop</p></td>
      <td><p>Optional</p></td>
      <td><p>Text to stop the inference. The output result does not include the stop words by default.</p></td>

      <td>
        <p>List\[string] type or string type, default value null.</p>

        <ul>
          <li>For List\[string], the number of elements should not exceed 1024, with each element having a length of 1-1024. The total length of the list elements should not exceed 32768 (256\*128). An empty list is equivalent to null.</li>
          <li>For string type, the length range is 1\~1024 characters.</li>
        </ul>

        <p>PD-separated scenarios do not support this parameter.</p>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>stop\_token\_ids</p></td>
      <td><p>Optional</p></td>
      <td><p>List of token IDs to stop the inference. The output result does not include the token IDs in the stop inference list by default.</p></td>
      <td><p>List\[int32] type, elements exceeding int32 will be ignored, default value is null.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>include\_stop\_str\_in\_output</p></td>
      <td><p>Optional</p></td>
      <td><p>Determines whether to include the stop string in the generated inference text.</p></td>

      <td>
        <p>Bool type, default value is false.</p>

        <ul>
          <li>true: Includes stop string.</li>
          <li>false: Does not include stop string.</li>
        </ul>

        <p>If stop or stop\_token\_ids is not provided, this field will be ignored.</p>
        <p>PD-separated scenarios do not support this parameter.</p>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>skip\_special\_tokens</p></td>
      <td><p>Optional</p></td>
      <td><p>Specifies whether to skip special tokens in the generated inference text.</p></td>

      <td>
        <p>Bool type, default value is true.</p>

        <ul>
          <li>true: Skip special tokens.</li>
          <li>false: Retain special tokens.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>ignore\_eos</p></td>
      <td><p>Optional</p></td>
      <td><p>Specifies whether to ignore the eos\_token end symbol during inference text generation.</p></td>

      <td>
        <p>Bool type, default value is false.</p>

        <ul>
          <li>true: Ignore eos\_token end symbol.</li>
          <li>false: Do not ignore eos\_token end symbol.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>max\_tokens</p></td>
      <td><p>Optional</p></td>
      <td><p>Specifies the maximum number of tokens allowed in the generated inference. The actual number of tokens generated is also affected by the maxIterTimes parameter in the configuration file, and the generated token count is less than or equal to min(maxIterTimes, max\_tokens).</p></td>
      <td><p>Int type, range (0, 2147483647], default value is maxIterTimes.</p></td>
    </tr>

    <tr>
      <td colspan="4"><p>tools</p></td>
      <td><p>Optional</p></td>
      <td><p>A list of tools that may be used.</p></td>
      <td><p>List\[dict] type.</p></td>
    </tr>

    <tr>
      <td rowspan="10"><p>-</p></td>
      <td colspan="3"><p>type</p></td>
      <td><p>Required</p></td>
      <td><p>Indicates the tool type.</p></td>
      <td><p>Only supports the string "function".</p></td>
    </tr>

    <tr>
      <td colspan="3"><p>function</p></td>
      <td><p>Required</p></td>
      <td><p>Function description.</p></td>
      <td><p>dict type.</p></td>
    </tr>

    <tr>
      <td rowspan="8"><p>-</p></td>
      <td colspan="2"><p>name</p></td>
      <td><p>Required</p></td>
      <td><p>Function name.</p></td>
      <td><p>String.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>strict</p></td>
      <td><p>Optional</p></td>
      <td><p>Indicates whether the generated tool calls strictly follow the schema format.</p></td>
      <td><p>bool type, default is false.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>description</p></td>
      <td><p>Optional</p></td>
      <td><p>Describes the function's functionality and usage.</p></td>
      <td><p>String.</p></td>
    </tr>

    <tr>
      <td colspan="2"><p>parameters</p></td>
      <td><p>Optional</p></td>
      <td><p>Indicates the parameters accepted by the function.</p></td>
      <td><p>JSON schema format.</p></td>
    </tr>

    <tr>
      <td rowspan="4"><p>-</p></td>
      <td><p>type</p></td>
      <td><p>Required</p></td>
      <td><p>Indicates the type of the function parameter's attribute.</p></td>
      <td><p>String, only supports "object".</p></td>
    </tr>

    <tr>
      <td><p>properties</p></td>
      <td><p>Required</p></td>
      <td><p>Properties of the function parameters. Each key represents a parameter name, which can be defined by the user. The value is of type dict, representing the parameter description, containing type and description fields.</p></td>
      <td><p>dict type.</p></td>
    </tr>

    <tr>
      <td><p>required</p></td>
      <td><p>Required</p></td>
      <td><p>Indicates the list of required parameters for the function.</p></td>
      <td><p>List\[string] type.</p></td>
    </tr>

    <tr>
      <td><p>additionalProperties</p></td>
      <td><p>Optional</p></td>
      <td><p>Indicates whether additional unmentioned parameters are allowed.</p></td>

      <td>
        <p>bool type, default value is false.</p>

        <ul>
          <li>true: Allows additional unmentioned parameters.</li>
          <li>false: Does not allow additional unmentioned parameters.</li>
        </ul>
      </td>
    </tr>

    <tr>
      <td colspan="4"><p>tool\_choice</p></td>
      <td><p>Optional</p></td>
      <td><p>Controls whether the model calls a tool.</p></td>

      <td>
        <p>string type or dict type, can be null, default value is "auto".</p>

        <ul>
          <li>"none": Indicates that the model will not call any tools and will generate a message instead.</li>
          <li>"auto": Indicates that the model can either generate a message or call one or more tools.</li>
          <li>"required": Indicates that the model must call one or more tools.</li>
        </ul>

        <p>By specifying {"{type: 'function', function: {name: 'my_function'}}"} you can specify a particular tool, forcing the model to call that tool.</p>
      </td>
    </tr>
  </tbody>
</table>