mirror of
https://github.com/QuantumNous/new-api.git
synced 2026-04-03 15:01:28 +00:00
Compare commits
17 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
d3ca454c3b | ||
|
|
46aca8fad3 | ||
|
|
86aeb72549 | ||
|
|
4dbdbdec1d | ||
|
|
36a739e777 | ||
|
|
98f92f990a | ||
|
|
3f7ea1fd83 | ||
|
|
f6e7a2344b | ||
|
|
3257723a55 | ||
|
|
b19b2d62df | ||
|
|
f9c8624f2c | ||
|
|
6c8253156b | ||
|
|
a66b314f5b | ||
|
|
e29ff0060d | ||
|
|
d4a2c2ab54 | ||
|
|
ded463ee57 | ||
|
|
e337936227 |
@@ -1,53 +0,0 @@
|
||||
# API 鉴权文档
|
||||
|
||||
## 认证方式
|
||||
|
||||
### Access Token
|
||||
|
||||
对于需要鉴权的 API 接口，必须同时提供以下两个请求头来进行 Access Token 认证：
|
||||
|
||||
1. **请求头中的 `Authorization` 字段**
|
||||
|
||||
将 Access Token 放置于 HTTP 请求头部的 `Authorization` 字段中，格式如下：
|
||||
|
||||
```
|
||||
Authorization: <your_access_token>
|
||||
```
|
||||
|
||||
其中 `<your_access_token>` 需要替换为实际的 Access Token 值。
|
||||
|
||||
2. **请求头中的 `New-Api-User` 字段**
|
||||
|
||||
将用户 ID 放置于 HTTP 请求头部的 `New-Api-User` 字段中，格式如下：
|
||||
|
||||
```
|
||||
New-Api-User: <your_user_id>
|
||||
```
|
||||
|
||||
其中 `<your_user_id>` 需要替换为实际的用户 ID。
|
||||
|
||||
**注意：**
|
||||
|
||||
* **必须同时提供 `Authorization` 和 `New-Api-User` 两个请求头才能通过鉴权。**
|
||||
* 如果只提供其中一个请求头，或者两个请求头都未提供，则会返回 `401 Unauthorized` 错误。
|
||||
* 如果 `Authorization` 中的 Access Token 无效，则会返回 `401 Unauthorized` 错误，并提示“无权进行此操作，access token 无效”。
|
||||
* 如果 `New-Api-User` 中的用户 ID 与 Access Token 不匹配，则会返回 `401 Unauthorized` 错误，并提示“无权进行此操作，与登录用户不匹配，请重新登录”。
|
||||
* 如果没有提供 `New-Api-User` 请求头，则会返回 `401 Unauthorized` 错误，并提示“无权进行此操作，未提供 New-Api-User”。
|
||||
* 如果 `New-Api-User` 请求头格式错误，则会返回 `401 Unauthorized` 错误，并提示“无权进行此操作，New-Api-User 格式错误”。
|
||||
* 如果用户已被禁用，则会返回 `403 Forbidden` 错误，并提示“用户已被封禁”。
|
||||
* 如果用户权限不足，则会返回 `403 Forbidden` 错误，并提示“无权进行此操作，权限不足”。
|
||||
* 如果用户信息无效，则会返回 `403 Forbidden` 错误，并提示“无权进行此操作，用户信息无效”。
|
||||
|
||||
## Curl 示例
|
||||
|
||||
假设您的 Access Token 为 `access_token`，用户 ID 为 `123`，要访问的 API 接口为 `/api/user/self`，则可以使用以下 curl 命令：
|
||||
|
||||
```bash
|
||||
curl -X GET \
|
||||
-H "Authorization: access_token" \
|
||||
-H "New-Api-User: 123" \
|
||||
https://your-domain.com/api/user/self
|
||||
```
|
||||
|
||||
请将 `access_token`、`123` 和 `https://your-domain.com` 替换为实际的值。
|
||||
|
||||
@@ -1,198 +0,0 @@
|
||||
# New API – Web 界面后端接口文档
|
||||
|
||||
> 本文档汇总了 **New API** 后端提供给前端 Web 界面的全部 REST 接口（不含 *Relay* 相关接口）。
|
||||
>
|
||||
> 接口前缀统一为 `https://<your-domain>`，以下仅列出 **路径**、**HTTP 方法**、**鉴权要求** 与 **功能简介**。
|
||||
>
|
||||
> 鉴权级别说明：
|
||||
> * **公开** – 不需要登录即可调用
|
||||
> * **用户** – 需携带用户 Token（`middleware.UserAuth`）
|
||||
> * **管理员** – 需管理员 Token（`middleware.AdminAuth`）
|
||||
> * **Root** – 仅限最高权限 Root 用户（`middleware.RootAuth`）
|
||||
|
||||
---
|
||||
|
||||
## 1. 初始化 / 系统状态
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/setup | 公开 | 获取系统初始化状态 |
|
||||
| POST | /api/setup | 公开 | 完成首次安装向导 |
|
||||
| GET | /api/status | 公开 | 获取运行状态摘要 |
|
||||
| GET | /api/uptime/status | 公开 | Uptime-Kuma 兼容状态探针 |
|
||||
| GET | /api/status/test | 管理员 | 测试后端与依赖组件是否正常 |
|
||||
|
||||
## 2. 公共信息
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/models | 用户 | 获取前端可用模型列表 |
|
||||
| GET | /api/notice | 公开 | 获取公告栏内容 |
|
||||
| GET | /api/about | 公开 | 关于页面信息 |
|
||||
| GET | /api/home_page_content | 公开 | 首页自定义内容 |
|
||||
| GET | /api/pricing | 可匿名/用户 | 价格与套餐信息 |
|
||||
| GET | /api/ratio_config | 公开 | 模型倍率配置（仅公开字段） |
|
||||
|
||||
## 3. 邮件 / 身份验证
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/verification | 公开 (限流) | 发送邮箱验证邮件 |
|
||||
| GET | /api/reset_password | 公开 (限流) | 发送重置密码邮件 |
|
||||
| POST | /api/user/reset | 公开 | 提交重置密码请求 |
|
||||
|
||||
## 4. OAuth / 第三方登录
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/oauth/github | 公开 | GitHub OAuth 跳转 |
|
||||
| GET | /api/oauth/discord | 公开 | Discord 通用 OAuth 跳转 |
|
||||
| GET | /api/oauth/oidc | 公开 | OIDC 通用 OAuth 跳转 |
|
||||
| GET | /api/oauth/linuxdo | 公开 | LinuxDo OAuth 跳转 |
|
||||
| GET | /api/oauth/wechat | 公开 | 微信扫码登录跳转 |
|
||||
| GET | /api/oauth/wechat/bind | 公开 | 微信账户绑定 |
|
||||
| GET | /api/oauth/email/bind | 公开 | 邮箱绑定 |
|
||||
| GET | /api/oauth/telegram/login | 公开 | Telegram 登录 |
|
||||
| GET | /api/oauth/telegram/bind | 公开 | Telegram 账户绑定 |
|
||||
| GET | /api/oauth/state | 公开 | 获取随机 state（防 CSRF） |
|
||||
|
||||
## 5. 用户模块
|
||||
### 5.1 账号注册/登录
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| POST | /api/user/register | 公开 | 注册新账号 |
|
||||
| POST | /api/user/login | 公开 | 用户登录 |
|
||||
| GET | /api/user/logout | 用户 | 退出登录 |
|
||||
| GET | /api/user/epay/notify | 公开 | Epay 支付回调 |
|
||||
| GET | /api/user/groups | 公开 | 列出所有分组（无鉴权版） |
|
||||
|
||||
### 5.2 用户自身操作 (需登录)
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/user/self/groups | 用户 | 获取自己所在分组 |
|
||||
| GET | /api/user/self | 用户 | 获取个人资料 |
|
||||
| GET | /api/user/models | 用户 | 获取模型可见性 |
|
||||
| PUT | /api/user/self | 用户 | 修改个人资料 |
|
||||
| DELETE | /api/user/self | 用户 | 注销账号 |
|
||||
| GET | /api/user/token | 用户 | 生成用户级别 Access Token |
|
||||
| GET | /api/user/aff | 用户 | 获取推广码信息 |
|
||||
| POST | /api/user/topup | 用户 | 余额直充 |
|
||||
| POST | /api/user/pay | 用户 | 提交支付订单 |
|
||||
| POST | /api/user/amount | 用户 | 余额支付 |
|
||||
| POST | /api/user/aff_transfer | 用户 | 推广额度转账 |
|
||||
| PUT | /api/user/setting | 用户 | 更新用户设置 |
|
||||
|
||||
### 5.3 管理员用户管理
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/user/ | 管理员 | 获取全部用户列表 |
|
||||
| GET | /api/user/search | 管理员 | 搜索用户 |
|
||||
| GET | /api/user/:id | 管理员 | 获取单个用户信息 |
|
||||
| POST | /api/user/ | 管理员 | 创建用户 |
|
||||
| POST | /api/user/manage | 管理员 | 冻结/重置等管理操作 |
|
||||
| PUT | /api/user/ | 管理员 | 更新用户 |
|
||||
| DELETE | /api/user/:id | 管理员 | 删除用户 |
|
||||
|
||||
## 6. 站点选项 (Root)
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/option/ | Root | 获取全局配置 |
|
||||
| PUT | /api/option/ | Root | 更新全局配置 |
|
||||
| POST | /api/option/rest_model_ratio | Root | 重置模型倍率 |
|
||||
| POST | /api/option/migrate_console_setting | Root | 迁移旧版控制台配置 |
|
||||
|
||||
## 7. 模型倍率同步 (Root)
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/ratio_sync/channels | Root | 获取可同步渠道列表 |
|
||||
| POST | /api/ratio_sync/fetch | Root | 从上游拉取倍率 |
|
||||
|
||||
## 8. 渠道管理 (管理员)
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| GET | /api/channel/ | 获取渠道列表 |
|
||||
| GET | /api/channel/search | 搜索渠道 |
|
||||
| GET | /api/channel/models | 查询渠道模型能力 |
|
||||
| GET | /api/channel/models_enabled | 查询启用模型能力 |
|
||||
| GET | /api/channel/:id | 获取单个渠道 |
|
||||
| GET | /api/channel/test | 批量测试渠道连通性 |
|
||||
| GET | /api/channel/test/:id | 单个渠道测试 |
|
||||
| GET | /api/channel/update_balance | 批量刷新余额 |
|
||||
| GET | /api/channel/update_balance/:id | 单个刷新余额 |
|
||||
| POST | /api/channel/ | 新增渠道 |
|
||||
| PUT | /api/channel/ | 更新渠道 |
|
||||
| DELETE | /api/channel/disabled | 删除已禁用渠道 |
|
||||
| POST | /api/channel/tag/disabled | 批量禁用标签渠道 |
|
||||
| POST | /api/channel/tag/enabled | 批量启用标签渠道 |
|
||||
| PUT | /api/channel/tag | 编辑渠道标签 |
|
||||
| DELETE | /api/channel/:id | 删除渠道 |
|
||||
| POST | /api/channel/batch | 批量删除渠道 |
|
||||
| POST | /api/channel/fix | 修复渠道能力表 |
|
||||
| GET | /api/channel/fetch_models/:id | 拉取单渠道模型 |
|
||||
| POST | /api/channel/fetch_models | 拉取全部渠道模型 |
|
||||
| POST | /api/channel/batch/tag | 批量设置渠道标签 |
|
||||
| GET | /api/channel/tag/models | 根据标签获取模型 |
|
||||
| POST | /api/channel/copy/:id | 复制渠道 |
|
||||
|
||||
## 9. Token 管理
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/token/ | 用户 | 获取全部 Token |
|
||||
| GET | /api/token/search | 用户 | 搜索 Token |
|
||||
| GET | /api/token/:id | 用户 | 获取单个 Token |
|
||||
| POST | /api/token/ | 用户 | 创建 Token |
|
||||
| PUT | /api/token/ | 用户 | 更新 Token |
|
||||
| DELETE | /api/token/:id | 用户 | 删除 Token |
|
||||
| POST | /api/token/batch | 用户 | 批量删除 Token |
|
||||
|
||||
## 10. 兑换码管理 (管理员)
|
||||
| 方法 | 路径 | 说明 |
|
||||
|------|------|------|
|
||||
| GET | /api/redemption/ | 获取兑换码列表 |
|
||||
| GET | /api/redemption/search | 搜索兑换码 |
|
||||
| GET | /api/redemption/:id | 获取单个兑换码 |
|
||||
| POST | /api/redemption/ | 创建兑换码 |
|
||||
| PUT | /api/redemption/ | 更新兑换码 |
|
||||
| DELETE | /api/redemption/invalid | 删除无效兑换码 |
|
||||
| DELETE | /api/redemption/:id | 删除兑换码 |
|
||||
|
||||
## 11. 日志
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/log/ | 管理员 | 获取全部日志 |
|
||||
| DELETE | /api/log/ | 管理员 | 删除历史日志 |
|
||||
| GET | /api/log/stat | 管理员 | 日志统计 |
|
||||
| GET | /api/log/self/stat | 用户 | 我的日志统计 |
|
||||
| GET | /api/log/search | 管理员 | 搜索全部日志 |
|
||||
| GET | /api/log/self | 用户 | 获取我的日志 |
|
||||
| GET | /api/log/self/search | 用户 | 搜索我的日志 |
|
||||
| GET | /api/log/token | 公开 | 根据 Token 查询日志（支持 CORS） |
|
||||
|
||||
## 12. 数据统计
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/data/ | 管理员 | 全站用量按日期统计 |
|
||||
| GET | /api/data/self | 用户 | 我的用量按日期统计 |
|
||||
|
||||
## 13. 分组
|
||||
| GET | /api/group/ | 管理员 | 获取全部分组列表 |
|
||||
|
||||
## 14. Midjourney 任务
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/mj/self | 用户 | 获取自己的 MJ 任务 |
|
||||
| GET | /api/mj/ | 管理员 | 获取全部 MJ 任务 |
|
||||
|
||||
## 15. 任务中心
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /api/task/self | 用户 | 获取我的任务 |
|
||||
| GET | /api/task/ | 管理员 | 获取全部任务 |
|
||||
|
||||
## 16. 账户计费面板 (Dashboard)
|
||||
| 方法 | 路径 | 鉴权 | 说明 |
|
||||
|------|------|------|------|
|
||||
| GET | /dashboard/billing/subscription | 用户 Token | 获取订阅额度信息 |
|
||||
| GET | /v1/dashboard/billing/subscription | 同上 | 兼容 OpenAI SDK 路径 |
|
||||
| GET | /dashboard/billing/usage | 用户 Token | 获取使用量信息 |
|
||||
| GET | /v1/dashboard/billing/usage | 同上 | 兼容 OpenAI SDK 路径 |
|
||||
|
||||
---
|
||||
|
||||
> **更新日期**：2025.07.17
|
||||
@@ -1,82 +0,0 @@
|
||||
# Midjourney Proxy API文档
|
||||
|
||||
**简介**:Midjourney Proxy API文档
|
||||
|
||||
## 接口列表
|
||||
支持的接口如下：
|
||||
+ [x] /mj/submit/imagine
|
||||
+ [x] /mj/submit/change
|
||||
+ [x] /mj/submit/blend
|
||||
+ [x] /mj/submit/describe
|
||||
+ [x] /mj/image/{id} （通过此接口获取图片，**请必须在系统设置中填写服务器地址！！**）
|
||||
+ [x] /mj/task/{id}/fetch （此接口返回的图片地址为经过One API转发的地址）
|
||||
+ [x] /task/list-by-condition
|
||||
+ [x] /mj/submit/action （仅midjourney-proxy-plus支持，下同）
|
||||
+ [x] /mj/submit/modal
|
||||
+ [x] /mj/submit/shorten
|
||||
+ [x] /mj/task/{id}/image-seed
|
||||
+ [x] /mj/insight-face/swap （InsightFace）
|
||||
|
||||
## 模型列表
|
||||
|
||||
### midjourney-proxy支持
|
||||
|
||||
- mj_imagine (绘图)
|
||||
- mj_variation (变换)
|
||||
- mj_reroll (重绘)
|
||||
- mj_blend (混合)
|
||||
- mj_upscale (放大)
|
||||
- mj_describe (图生文)
|
||||
|
||||
### 仅midjourney-proxy-plus支持
|
||||
|
||||
- mj_zoom (比例变焦)
|
||||
- mj_shorten (提示词缩短)
|
||||
- mj_modal (窗口提交，局部重绘和自定义比例变焦必须和mj_modal一同添加)
|
||||
- mj_inpaint (局部重绘提交，必须和mj_modal一同添加)
|
||||
- mj_custom_zoom (自定义比例变焦，必须和mj_modal一同添加)
|
||||
- mj_high_variation (强变换)
|
||||
- mj_low_variation (弱变换)
|
||||
- mj_pan (平移)
|
||||
- swap_face (换脸)
|
||||
|
||||
## 模型价格设置（在设置-运营设置-模型固定价格设置中设置）
|
||||
```json
|
||||
{
|
||||
"mj_imagine": 0.1,
|
||||
"mj_variation": 0.1,
|
||||
"mj_reroll": 0.1,
|
||||
"mj_blend": 0.1,
|
||||
"mj_modal": 0.1,
|
||||
"mj_zoom": 0.1,
|
||||
"mj_shorten": 0.1,
|
||||
"mj_high_variation": 0.1,
|
||||
"mj_low_variation": 0.1,
|
||||
"mj_pan": 0.1,
|
||||
"mj_inpaint": 0,
|
||||
"mj_custom_zoom": 0,
|
||||
"mj_describe": 0.05,
|
||||
"mj_upscale": 0.05,
|
||||
"swap_face": 0.05
|
||||
}
|
||||
```
|
||||
其中mj_inpaint和mj_custom_zoom的价格设置为0，是因为这两个模型需要搭配mj_modal使用，所以价格由mj_modal决定。
|
||||
|
||||
## 渠道设置
|
||||
|
||||
### 对接 midjourney-proxy(plus)
|
||||
|
||||
1.
|
||||
|
||||
部署Midjourney-Proxy，并配置好midjourney账号等（强烈建议设置密钥），[项目地址](https://github.com/novicezk/midjourney-proxy)
|
||||
|
||||
2. 在渠道管理中添加渠道，渠道类型选择**Midjourney Proxy**，如果是plus版本选择**Midjourney Proxy Plus**
|
||||
，模型请参考上方模型列表
|
||||
3. **代理**填写midjourney-proxy部署的地址，例如：http://localhost:8080
|
||||
4. 密钥填写midjourney-proxy的密钥，如果没有设置密钥，可以随便填
|
||||
|
||||
### 对接上游new api
|
||||
|
||||
1. 在渠道管理中添加渠道，渠道类型选择**Midjourney Proxy Plus**，模型请参考上方模型列表
|
||||
2. **代理**填写上游new api的地址，例如：http://localhost:3000
|
||||
3. 密钥填写上游new api的密钥
|
||||
@@ -1,62 +0,0 @@
|
||||
# Rerank API文档
|
||||
|
||||
**简介**:Rerank API文档
|
||||
|
||||
## 接入Dify
|
||||
模型供应商选择Jina，按要求填写模型信息即可接入Dify。
|
||||
|
||||
## 请求方式
|
||||
|
||||
Post: /v1/rerank
|
||||
|
||||
Request:
|
||||
|
||||
```json
|
||||
{
|
||||
"model": "jina-reranker-v2-base-multilingual",
|
||||
"query": "What is the capital of the United States?",
|
||||
"top_n": 3,
|
||||
"documents": [
|
||||
"Carson City is the capital city of the American state of Nevada.",
|
||||
"The Commonwealth of the Northern Mariana Islands is a group of islands in the Pacific Ocean. Its capital is Saipan.",
|
||||
"Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district.",
|
||||
"Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages.",
|
||||
"Capital punishment (the death penalty) has existed in the United States since beforethe United States was a country. As of 2017, capital punishment is legal in 30 of the 50 states."
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
Response:
|
||||
|
||||
```json
|
||||
{
|
||||
"results": [
|
||||
{
|
||||
"document": {
|
||||
"text": "Washington, D.C. (also known as simply Washington or D.C., and officially as the District of Columbia) is the capital of the United States. It is a federal district."
|
||||
},
|
||||
"index": 2,
|
||||
"relevance_score": 0.9999702
|
||||
},
|
||||
{
|
||||
"document": {
|
||||
"text": "Carson City is the capital city of the American state of Nevada."
|
||||
},
|
||||
"index": 0,
|
||||
"relevance_score": 0.67800725
|
||||
},
|
||||
{
|
||||
"document": {
|
||||
"text": "Capitalization or capitalisation in English grammar is the use of a capital letter at the start of a word. English usage varies from capitalization in other languages."
|
||||
},
|
||||
"index": 3,
|
||||
"relevance_score": 0.02800752
|
||||
}
|
||||
],
|
||||
"usage": {
|
||||
"prompt_tokens": 158,
|
||||
"completion_tokens": 0,
|
||||
"total_tokens": 158
|
||||
}
|
||||
}
|
||||
```
|
||||
@@ -1,44 +0,0 @@
|
||||
# Suno API文档
|
||||
|
||||
**简介**:Suno API文档
|
||||
|
||||
## 接口列表
|
||||
支持的接口如下：
|
||||
+ [x] /suno/submit/music
|
||||
+ [x] /suno/submit/lyrics
|
||||
+ [x] /suno/fetch
|
||||
+ [x] /suno/fetch/:id
|
||||
|
||||
## 模型列表
|
||||
|
||||
### Suno API支持
|
||||
|
||||
- suno_music (自定义模式、灵感模式、续写)
|
||||
- suno_lyrics (生成歌词)
|
||||
|
||||
|
||||
## 模型价格设置（在设置-运营设置-模型固定价格设置中设置）
|
||||
```json
|
||||
{
|
||||
"suno_music": 0.3,
|
||||
"suno_lyrics": 0.01
|
||||
}
|
||||
```
|
||||
|
||||
## 渠道设置
|
||||
|
||||
### 对接 Suno API
|
||||
|
||||
1.
|
||||
部署 Suno API，并配置好suno账号等（强烈建议设置密钥），[项目地址](https://github.com/Suno-API/Suno-API)
|
||||
|
||||
2. 在渠道管理中添加渠道，渠道类型选择**Suno API**
|
||||
，模型请参考上方模型列表
|
||||
3. **代理**填写 Suno API 部署的地址，例如：http://localhost:8080
|
||||
4. 密钥填写 Suno API 的密钥，如果没有设置密钥，可以随便填
|
||||
|
||||
### 对接上游new api
|
||||
|
||||
1. 在渠道管理中添加渠道，渠道类型选择**Suno API**，或任意类型，只需模型包含上方模型列表的模型
|
||||
2. **代理**填写上游new api的地址，例如：http://localhost:3000
|
||||
3. 密钥填写上游new api的密钥
|
||||
7818
docs/openapi/api.json
Normal file
7818
docs/openapi/api.json
Normal file
File diff suppressed because it is too large
Load Diff
7141
docs/openapi/relay.json
Normal file
7141
docs/openapi/relay.json
Normal file
File diff suppressed because it is too large
Load Diff
@@ -169,7 +169,7 @@ type ImageResponse struct {
|
||||
Extra any `json:"extra,omitempty"`
|
||||
}
|
||||
type ImageData struct {
|
||||
Url string `json:"url,omitempty"`
|
||||
B64Json string `json:"b64_json,omitempty"`
|
||||
RevisedPrompt string `json:"revised_prompt,omitempty"`
|
||||
Url string `json:"url"`
|
||||
B64Json string `json:"b64_json"`
|
||||
RevisedPrompt string `json:"revised_prompt"`
|
||||
}
|
||||
|
||||
@@ -27,8 +27,6 @@ import (
|
||||
func SetupApiRequestHeader(info *common.RelayInfo, c *gin.Context, req *http.Header) {
|
||||
if info.RelayMode == constant.RelayModeAudioTranscription || info.RelayMode == constant.RelayModeAudioTranslation {
|
||||
// multipart/form-data
|
||||
} else if info.RelayMode == constant.RelayModeImagesEdits {
|
||||
// multipart/form-data
|
||||
} else if info.RelayMode == constant.RelayModeRealtime {
|
||||
// websocket
|
||||
} else {
|
||||
|
||||
@@ -1,12 +1,10 @@
|
||||
package gemini
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"slices"
|
||||
"strings"
|
||||
|
||||
"github.com/QuantumNous/new-api/dto"
|
||||
@@ -57,139 +55,9 @@ func (a *Adaptor) ConvertAudioRequest(c *gin.Context, info *relaycommon.RelayInf
|
||||
return nil, errors.New("not implemented")
|
||||
}
|
||||
|
||||
type ImageConfig struct {
|
||||
AspectRatio string `json:"aspectRatio,omitempty"`
|
||||
ImageSize string `json:"imageSize,omitempty"`
|
||||
}
|
||||
|
||||
type SizeMapping struct {
|
||||
AspectRatio string
|
||||
ImageSize string
|
||||
}
|
||||
|
||||
type QualityMapping struct {
|
||||
Standard string
|
||||
HD string
|
||||
High string
|
||||
FourK string
|
||||
Auto string
|
||||
}
|
||||
|
||||
func getImageSizeMapping() QualityMapping {
|
||||
return QualityMapping{
|
||||
Standard: "1K",
|
||||
HD: "2K",
|
||||
High: "2K",
|
||||
FourK: "4K",
|
||||
Auto: "1K",
|
||||
}
|
||||
}
|
||||
|
||||
func getSizeMappings() map[string]SizeMapping {
|
||||
return map[string]SizeMapping{
|
||||
// Gemini 2.5 Flash Image - default 1K resolutions
|
||||
"1024x1024": {AspectRatio: "1:1", ImageSize: ""},
|
||||
"832x1248": {AspectRatio: "2:3", ImageSize: ""},
|
||||
"1248x832": {AspectRatio: "3:2", ImageSize: ""},
|
||||
"864x1184": {AspectRatio: "3:4", ImageSize: ""},
|
||||
"1184x864": {AspectRatio: "4:3", ImageSize: ""},
|
||||
"896x1152": {AspectRatio: "4:5", ImageSize: ""},
|
||||
"1152x896": {AspectRatio: "5:4", ImageSize: ""},
|
||||
"768x1344": {AspectRatio: "9:16", ImageSize: ""},
|
||||
"1344x768": {AspectRatio: "16:9", ImageSize: ""},
|
||||
"1536x672": {AspectRatio: "21:9", ImageSize: ""},
|
||||
|
||||
// Gemini 3 Pro Image Preview resolutions
|
||||
"1536x1024": {AspectRatio: "3:2", ImageSize: ""},
|
||||
"1024x1536": {AspectRatio: "2:3", ImageSize: ""},
|
||||
"1024x1792": {AspectRatio: "9:16", ImageSize: ""},
|
||||
"1792x1024": {AspectRatio: "16:9", ImageSize: ""},
|
||||
"2048x2048": {AspectRatio: "1:1", ImageSize: "2K"},
|
||||
"4096x4096": {AspectRatio: "1:1", ImageSize: "4K"},
|
||||
}
|
||||
}
|
||||
|
||||
func processSizeParameters(size, quality string) ImageConfig {
|
||||
config := ImageConfig{} // 默认为空值
|
||||
|
||||
if size != "" {
|
||||
if strings.Contains(size, ":") {
|
||||
config.AspectRatio = size // 直接设置，不与默认值比较
|
||||
} else {
|
||||
if mapping, exists := getSizeMappings()[size]; exists {
|
||||
if mapping.AspectRatio != "" {
|
||||
config.AspectRatio = mapping.AspectRatio
|
||||
}
|
||||
if mapping.ImageSize != "" {
|
||||
config.ImageSize = mapping.ImageSize
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if quality != "" {
|
||||
qualityMapping := getImageSizeMapping()
|
||||
switch strings.ToLower(strings.TrimSpace(quality)) {
|
||||
case "hd", "high":
|
||||
config.ImageSize = qualityMapping.HD
|
||||
case "4k":
|
||||
config.ImageSize = qualityMapping.FourK
|
||||
case "standard", "medium", "low", "auto", "1k":
|
||||
config.ImageSize = qualityMapping.Standard
|
||||
}
|
||||
}
|
||||
|
||||
return config
|
||||
}
|
||||
|
||||
func (a *Adaptor) ConvertImageRequest(c *gin.Context, info *relaycommon.RelayInfo, request dto.ImageRequest) (any, error) {
|
||||
if model_setting.IsGeminiModelSupportImagine(info.UpstreamModelName) {
|
||||
var content any
|
||||
if base64Data, err := relaycommon.GetImageBase64sFromForm(c); err == nil {
|
||||
content = []any{
|
||||
dto.MediaContent{
|
||||
Type: dto.ContentTypeText,
|
||||
Text: request.Prompt,
|
||||
},
|
||||
dto.MediaContent{
|
||||
Type: dto.ContentTypeFile,
|
||||
File: &dto.MessageFile{
|
||||
FileData: base64Data.String(),
|
||||
},
|
||||
},
|
||||
}
|
||||
} else {
|
||||
content = request.Prompt
|
||||
}
|
||||
|
||||
chatRequest := dto.GeneralOpenAIRequest{
|
||||
Model: request.Model,
|
||||
Messages: []dto.Message{
|
||||
{Role: "user", Content: content},
|
||||
},
|
||||
N: int(request.N),
|
||||
}
|
||||
|
||||
config := processSizeParameters(strings.TrimSpace(request.Size), request.Quality)
|
||||
|
||||
// 兼容 nano-banana 传quality[imageSize]会报错 An internal error has occurred. Please retry or report in https://developers.generativeai.google/guide/troubleshooting
|
||||
if slices.Contains([]string{"nano-banana", "gemini-2.5-flash-image"}, info.UpstreamModelName) {
|
||||
config.ImageSize = ""
|
||||
}
|
||||
|
||||
googleGenerationConfig := map[string]interface{}{
|
||||
"responseModalities": []string{"TEXT", "IMAGE"},
|
||||
"imageConfig": config,
|
||||
}
|
||||
|
||||
extraBody := map[string]interface{}{
|
||||
"google": map[string]interface{}{
|
||||
"generationConfig": googleGenerationConfig,
|
||||
},
|
||||
}
|
||||
chatRequest.ExtraBody, _ = json.Marshal(extraBody)
|
||||
|
||||
return a.ConvertOpenAIRequest(c, info, &chatRequest)
|
||||
if !strings.HasPrefix(info.UpstreamModelName, "imagen") {
|
||||
return nil, errors.New("not supported model for image generation")
|
||||
}
|
||||
|
||||
// convert size to aspect ratio but allow user to specify aspect ratio
|
||||
@@ -199,8 +67,17 @@ func (a *Adaptor) ConvertImageRequest(c *gin.Context, info *relaycommon.RelayInf
|
||||
if strings.Contains(size, ":") {
|
||||
aspectRatio = size
|
||||
} else {
|
||||
if mapping, exists := getSizeMappings()[size]; exists && mapping.AspectRatio != "" {
|
||||
aspectRatio = mapping.AspectRatio
|
||||
switch size {
|
||||
case "256x256", "512x512", "1024x1024":
|
||||
aspectRatio = "1:1"
|
||||
case "1536x1024":
|
||||
aspectRatio = "3:2"
|
||||
case "1024x1536":
|
||||
aspectRatio = "2:3"
|
||||
case "1024x1792":
|
||||
aspectRatio = "9:16"
|
||||
case "1792x1024":
|
||||
aspectRatio = "16:9"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -381,10 +258,6 @@ func (a *Adaptor) DoResponse(c *gin.Context, resp *http.Response, info *relaycom
|
||||
return GeminiImageHandler(c, info, resp)
|
||||
}
|
||||
|
||||
if model_setting.IsGeminiModelSupportImagine(info.UpstreamModelName) {
|
||||
return ChatImageHandler(c, info, resp)
|
||||
}
|
||||
|
||||
// check if the model is an embedding model
|
||||
if strings.HasPrefix(info.UpstreamModelName, "text-embedding") ||
|
||||
strings.HasPrefix(info.UpstreamModelName, "embedding") ||
|
||||
|
||||
@@ -183,7 +183,7 @@ func ThinkingAdaptor(geminiRequest *dto.GeminiChatRequest, info *relaycommon.Rel
|
||||
}
|
||||
|
||||
// Setting safety to the lowest possible values since Gemini is already powerless enough
|
||||
func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, info *relaycommon.RelayInfo, base64Data ...*relaycommon.Base64Data) (*dto.GeminiChatRequest, error) {
|
||||
func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, info *relaycommon.RelayInfo) (*dto.GeminiChatRequest, error) {
|
||||
|
||||
geminiRequest := dto.GeminiChatRequest{
|
||||
Contents: make([]dto.GeminiChatContent, 0, len(textRequest.Messages)),
|
||||
@@ -208,6 +208,7 @@ func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, i
|
||||
|
||||
adaptorWithExtraBody := false
|
||||
|
||||
// patch extra_body
|
||||
if len(textRequest.ExtraBody) > 0 {
|
||||
if !strings.HasSuffix(info.UpstreamModelName, "-nothinking") {
|
||||
var extraBody map[string]interface{}
|
||||
@@ -240,13 +241,36 @@ func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, i
|
||||
}
|
||||
}
|
||||
|
||||
if generationConfig, ok := googleBody["generationConfig"].(map[string]any); ok {
|
||||
generationConfigBytes, err := json.Marshal(generationConfig)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to marshal generationConfig: %w", err)
|
||||
// check error param name like imageConfig, should be image_config
|
||||
if _, hasErrorParam := googleBody["imageConfig"]; hasErrorParam {
|
||||
return nil, errors.New("extra_body.google.imageConfig is not supported, use extra_body.google.image_config instead")
|
||||
}
|
||||
|
||||
if imageConfig, ok := googleBody["image_config"].(map[string]interface{}); ok {
|
||||
// check error param name like aspectRatio, should be aspect_ratio
|
||||
if _, hasErrorParam := imageConfig["aspectRatio"]; hasErrorParam {
|
||||
return nil, errors.New("extra_body.google.image_config.aspectRatio is not supported, use extra_body.google.image_config.aspect_ratio instead")
|
||||
}
|
||||
if err := json.Unmarshal(generationConfigBytes, &geminiRequest.GenerationConfig); err != nil {
|
||||
return nil, fmt.Errorf("failed to unmarshal generationConfig: %w", err)
|
||||
// check error param name like imageSize, should be image_size
|
||||
if _, hasErrorParam := imageConfig["imageSize"]; hasErrorParam {
|
||||
return nil, errors.New("extra_body.google.image_config.imageSize is not supported, use extra_body.google.image_config.image_size instead")
|
||||
}
|
||||
|
||||
// convert snake_case to camelCase for Gemini API
|
||||
geminiImageConfig := make(map[string]interface{})
|
||||
if aspectRatio, ok := imageConfig["aspect_ratio"]; ok {
|
||||
geminiImageConfig["aspectRatio"] = aspectRatio
|
||||
}
|
||||
if imageSize, ok := imageConfig["image_size"]; ok {
|
||||
geminiImageConfig["imageSize"] = imageSize
|
||||
}
|
||||
|
||||
if len(geminiImageConfig) > 0 {
|
||||
imageConfigBytes, err := common.Marshal(geminiImageConfig)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("failed to marshal image_config: %w", err)
|
||||
}
|
||||
geminiRequest.GenerationConfig.ImageConfig = imageConfigBytes
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -422,9 +446,68 @@ func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, i
|
||||
if part.Text == "" {
|
||||
continue
|
||||
}
|
||||
parts = append(parts, dto.GeminiPart{
|
||||
Text: part.Text,
|
||||
})
|
||||
// check markdown image 
|
||||
// 使用字符串查找而非正则，避免大文本性能问题
|
||||
text := part.Text
|
||||
hasMarkdownImage := false
|
||||
for {
|
||||
// 快速检查是否包含 markdown 图片标记
|
||||
startIdx := strings.Index(text, "
|
||||
if bracketIdx == -1 {
|
||||
break
|
||||
}
|
||||
bracketIdx += startIdx
|
||||
// 找到闭合的 )
|
||||
closeIdx := strings.Index(text[bracketIdx+2:], ")")
|
||||
if closeIdx == -1 {
|
||||
break
|
||||
}
|
||||
closeIdx += bracketIdx + 2
|
||||
|
||||
hasMarkdownImage = true
|
||||
// 添加图片前的文本
|
||||
if startIdx > 0 {
|
||||
textBefore := text[:startIdx]
|
||||
if textBefore != "" {
|
||||
parts = append(parts, dto.GeminiPart{
|
||||
Text: textBefore,
|
||||
})
|
||||
}
|
||||
}
|
||||
// 提取 data URL (从 "](" 后面开始，到 ")" 之前)
|
||||
dataUrl := text[bracketIdx+2 : closeIdx]
|
||||
imageNum += 1
|
||||
if constant.GeminiVisionMaxImageNum != -1 && imageNum > constant.GeminiVisionMaxImageNum {
|
||||
return nil, fmt.Errorf("too many images in the message, max allowed is %d", constant.GeminiVisionMaxImageNum)
|
||||
}
|
||||
format, base64String, err := service.DecodeBase64FileData(dataUrl)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode markdown base64 image data failed: %s", err.Error())
|
||||
}
|
||||
imgPart := dto.GeminiPart{
|
||||
InlineData: &dto.GeminiInlineData{
|
||||
MimeType: format,
|
||||
Data: base64String,
|
||||
},
|
||||
}
|
||||
if shouldAttachThoughtSignature {
|
||||
imgPart.ThoughtSignature = json.RawMessage(strconv.Quote(thoughtSignatureBypassValue))
|
||||
}
|
||||
parts = append(parts, imgPart)
|
||||
// 继续处理剩余文本
|
||||
text = text[closeIdx+1:]
|
||||
}
|
||||
// 添加剩余文本或原始文本（如果没有找到 markdown 图片）
|
||||
if !hasMarkdownImage {
|
||||
parts = append(parts, dto.GeminiPart{
|
||||
Text: part.Text,
|
||||
})
|
||||
}
|
||||
} else if part.Type == dto.ContentTypeImageURL {
|
||||
imageNum += 1
|
||||
|
||||
@@ -464,11 +547,10 @@ func CovertOpenAI2Gemini(c *gin.Context, textRequest dto.GeneralOpenAIRequest, i
|
||||
})
|
||||
}
|
||||
} else if part.Type == dto.ContentTypeFile {
|
||||
file := part.GetFile()
|
||||
if file.FileId != "" {
|
||||
if part.GetFile().FileId != "" {
|
||||
return nil, fmt.Errorf("only base64 file is supported in gemini")
|
||||
}
|
||||
format, base64String, err := service.DecodeBase64FileData(file.FileData)
|
||||
format, base64String, err := service.DecodeBase64FileData(part.GetFile().FileData)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decode base64 file data failed: %s", err.Error())
|
||||
}
|
||||
@@ -1275,70 +1357,3 @@ func GeminiImageHandler(c *gin.Context, info *relaycommon.RelayInfo, resp *http.
|
||||
|
||||
return usage, nil
|
||||
}
|
||||
|
||||
func convertToOaiImageResponse(geminiResponse *dto.GeminiChatResponse) (*dto.ImageResponse, error) {
|
||||
openAIResponse := &dto.ImageResponse{
|
||||
Created: common.GetTimestamp(),
|
||||
Data: make([]dto.ImageData, 0),
|
||||
}
|
||||
|
||||
// extract images from candidates' inlineData
|
||||
for _, candidate := range geminiResponse.Candidates {
|
||||
for _, part := range candidate.Content.Parts {
|
||||
if part.InlineData != nil && strings.HasPrefix(part.InlineData.MimeType, "image") {
|
||||
openAIResponse.Data = append(openAIResponse.Data, dto.ImageData{
|
||||
B64Json: part.InlineData.Data,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if len(openAIResponse.Data) == 0 {
|
||||
return nil, errors.New("no images found in response")
|
||||
}
|
||||
|
||||
return openAIResponse, nil
|
||||
}
|
||||
|
||||
func ChatImageHandler(c *gin.Context, info *relaycommon.RelayInfo, resp *http.Response) (*dto.Usage, *types.NewAPIError) {
|
||||
responseBody, readErr := io.ReadAll(resp.Body)
|
||||
if readErr != nil {
|
||||
return nil, types.NewOpenAIError(readErr, types.ErrorCodeBadResponseBody, http.StatusInternalServerError)
|
||||
}
|
||||
service.CloseResponseBodyGracefully(resp)
|
||||
|
||||
if common.DebugEnabled {
|
||||
println("ChatImageHandler response:", string(responseBody))
|
||||
}
|
||||
|
||||
var geminiResponse dto.GeminiChatResponse
|
||||
if jsonErr := common.Unmarshal(responseBody, &geminiResponse); jsonErr != nil {
|
||||
return nil, types.NewOpenAIError(jsonErr, types.ErrorCodeBadResponseBody, http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
if len(geminiResponse.Candidates) == 0 {
|
||||
return nil, types.NewOpenAIError(errors.New("no images generated"), types.ErrorCodeBadResponseBody, http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
openAIResponse, err := convertToOaiImageResponse(&geminiResponse)
|
||||
if err != nil {
|
||||
return nil, types.NewOpenAIError(err, types.ErrorCodeBadResponseBody, http.StatusInternalServerError)
|
||||
}
|
||||
|
||||
jsonResponse, jsonErr := json.Marshal(openAIResponse)
|
||||
if jsonErr != nil {
|
||||
return nil, types.NewError(jsonErr, types.ErrorCodeBadResponseBody)
|
||||
}
|
||||
|
||||
c.Writer.Header().Set("Content-Type", "application/json")
|
||||
c.Writer.WriteHeader(resp.StatusCode)
|
||||
_, _ = c.Writer.Write(jsonResponse)
|
||||
|
||||
usage := &dto.Usage{
|
||||
PromptTokens: geminiResponse.UsageMetadata.PromptTokenCount,
|
||||
CompletionTokens: geminiResponse.UsageMetadata.CandidatesTokenCount,
|
||||
TotalTokens: geminiResponse.UsageMetadata.TotalTokenCount,
|
||||
}
|
||||
|
||||
return usage, nil
|
||||
}
|
||||
|
||||
@@ -24,9 +24,13 @@ import (
|
||||
"github.com/pkg/errors"
|
||||
)
|
||||
|
||||
// VideoGenerationConfig represents the video generation configuration
|
||||
// ============================
|
||||
// Request / Response structures
|
||||
// ============================
|
||||
|
||||
// GeminiVideoGenerationConfig represents the video generation configuration
|
||||
// Based on: https://ai.google.dev/gemini-api/docs/video
|
||||
type VideoGenerationConfig struct {
|
||||
type GeminiVideoGenerationConfig struct {
|
||||
AspectRatio string `json:"aspectRatio,omitempty"` // "16:9" or "9:16"
|
||||
DurationSeconds float64 `json:"durationSeconds,omitempty"` // 4, 6, or 8 (as number)
|
||||
NegativePrompt string `json:"negativePrompt,omitempty"` // unwanted elements
|
||||
@@ -34,21 +38,15 @@ type VideoGenerationConfig struct {
|
||||
Resolution string `json:"resolution,omitempty"` // video resolution
|
||||
}
|
||||
|
||||
type Image struct {
|
||||
BytesBase64Encoded string `json:"bytesBase64Encoded,omitempty"`
|
||||
MimeType string `json:"mimeType,omitempty"`
|
||||
// GeminiVideoRequest represents a single video generation instance
|
||||
type GeminiVideoRequest struct {
|
||||
Prompt string `json:"prompt"`
|
||||
}
|
||||
|
||||
type VideoRequest struct {
|
||||
Prompt string `json:"prompt"`
|
||||
Image *Image `json:"image,omitempty"`
|
||||
LastFrame *Image `json:"lastFrame,omitempty"`
|
||||
}
|
||||
|
||||
// VideoPayload represents the complete video generation request payload
|
||||
type VideoPayload struct {
|
||||
Instances []VideoRequest `json:"instances"`
|
||||
Parameters VideoGenerationConfig `json:"parameters,omitempty"`
|
||||
// GeminiVideoPayload represents the complete video generation request payload
|
||||
type GeminiVideoPayload struct {
|
||||
Instances []GeminiVideoRequest `json:"instances"`
|
||||
Parameters GeminiVideoGenerationConfig `json:"parameters,omitempty"`
|
||||
}
|
||||
|
||||
type submitResponse struct {
|
||||
@@ -77,8 +75,6 @@ type operationResponse struct {
|
||||
URI string `json:"uri"`
|
||||
} `json:"video"`
|
||||
} `json:"generatedSamples"`
|
||||
RaiMediaFilteredCount int `json:"raiMediaFilteredCount"`
|
||||
RaiMediaFilteredReasons []string `json:"raiMediaFilteredReasons"`
|
||||
} `json:"generateVideoResponse"`
|
||||
} `json:"response"`
|
||||
Error struct {
|
||||
@@ -104,7 +100,8 @@ func (a *TaskAdaptor) Init(info *relaycommon.RelayInfo) {
|
||||
|
||||
// ValidateRequestAndSetAction parses body, validates fields and sets default action.
|
||||
func (a *TaskAdaptor) ValidateRequestAndSetAction(c *gin.Context, info *relaycommon.RelayInfo) (taskErr *dto.TaskError) {
|
||||
return relaycommon.ValidateBasicTaskRequest(c, info, constant.TaskActionGenerate)
|
||||
// Use the standard validation method for TaskSubmitReq
|
||||
return relaycommon.ValidateBasicTaskRequest(c, info, constant.TaskActionTextGenerate)
|
||||
}
|
||||
|
||||
// BuildRequestURL constructs the upstream URL.
|
||||
@@ -140,21 +137,13 @@ func (a *TaskAdaptor) BuildRequestBody(c *gin.Context, info *relaycommon.RelayIn
|
||||
}
|
||||
|
||||
// Create structured video generation request
|
||||
body := VideoPayload{
|
||||
Instances: []VideoRequest{
|
||||
body := GeminiVideoPayload{
|
||||
Instances: []GeminiVideoRequest{
|
||||
{Prompt: req.Prompt},
|
||||
},
|
||||
Parameters: VideoGenerationConfig{},
|
||||
Parameters: GeminiVideoGenerationConfig{},
|
||||
}
|
||||
|
||||
if len(req.Images) > 0 {
|
||||
body.Instances[0].Image = a.convertImage(req.Images[0])
|
||||
}
|
||||
if len(req.Images) > 1 {
|
||||
body.Instances[0].LastFrame = a.convertImage(req.Images[1])
|
||||
}
|
||||
|
||||
// Parse metadata for additional configuration
|
||||
metadata := req.Metadata
|
||||
medaBytes, err := json.Marshal(metadata)
|
||||
if err != nil {
|
||||
@@ -258,19 +247,20 @@ func (a *TaskAdaptor) ParseTaskResult(respBody []byte) (*relaycommon.TaskInfo, e
|
||||
return ti, nil
|
||||
}
|
||||
|
||||
if len(op.Response.GenerateVideoResponse.GeneratedSamples) == 0 {
|
||||
ti.Status = model.TaskStatusFailure
|
||||
ti.Reason = fmt.Sprintf("no generated video url found: %s", strings.Join(op.Response.GenerateVideoResponse.RaiMediaFilteredReasons, "; "))
|
||||
} else {
|
||||
if uri := op.Response.GenerateVideoResponse.GeneratedSamples[0].Video.URI; uri != "" {
|
||||
ti.RemoteUrl = uri
|
||||
}
|
||||
ti.Status = model.TaskStatusSuccess
|
||||
}
|
||||
ti.Status = model.TaskStatusSuccess
|
||||
ti.Progress = "100%"
|
||||
|
||||
taskID := encodeLocalTaskID(op.Name)
|
||||
ti.TaskID = taskID
|
||||
ti.Url = fmt.Sprintf("%s/v1/videos/%s/content", system_setting.ServerAddress, taskID)
|
||||
|
||||
// Extract URL from generateVideoResponse if available
|
||||
if len(op.Response.GenerateVideoResponse.GeneratedSamples) > 0 {
|
||||
if uri := op.Response.GenerateVideoResponse.GeneratedSamples[0].Video.URI; uri != "" {
|
||||
ti.RemoteUrl = uri
|
||||
}
|
||||
}
|
||||
|
||||
return ti, nil
|
||||
}
|
||||
|
||||
@@ -299,30 +289,6 @@ func (a *TaskAdaptor) ConvertToOpenAIVideo(task *model.Task) ([]byte, error) {
|
||||
return common.Marshal(video)
|
||||
}
|
||||
|
||||
func (a *TaskAdaptor) convertImage(imageStr string) *Image {
|
||||
if strings.TrimSpace(imageStr) == "" {
|
||||
return nil
|
||||
}
|
||||
img := &Image{
|
||||
MimeType: "image/png",
|
||||
BytesBase64Encoded: imageStr,
|
||||
}
|
||||
if strings.HasPrefix(imageStr, "data:image/") {
|
||||
parts := strings.Split(imageStr, ";base64,")
|
||||
if len(parts) == 2 {
|
||||
img.MimeType = strings.TrimPrefix(parts[0], "data:")
|
||||
img.BytesBase64Encoded = parts[1]
|
||||
}
|
||||
} else if strings.HasPrefix(imageStr, "http") {
|
||||
mimeType, data, err := service.GetImageFromUrl(imageStr)
|
||||
if err == nil {
|
||||
img.MimeType = mimeType
|
||||
img.BytesBase64Encoded = data
|
||||
}
|
||||
}
|
||||
return img
|
||||
}
|
||||
|
||||
// ============================
|
||||
// helpers
|
||||
// ============================
|
||||
|
||||
@@ -12,7 +12,6 @@ import (
|
||||
|
||||
"github.com/QuantumNous/new-api/common"
|
||||
"github.com/QuantumNous/new-api/model"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
|
||||
"github.com/QuantumNous/new-api/constant"
|
||||
@@ -147,13 +146,40 @@ func (a *TaskAdaptor) BuildRequestBody(c *gin.Context, info *relaycommon.RelayIn
|
||||
body.Parameters["storageUri"] = v
|
||||
}
|
||||
if v, ok := req.Metadata["sampleCount"]; ok {
|
||||
body.Parameters["sampleCount"] = v
|
||||
if i, ok := v.(int); ok {
|
||||
body.Parameters["sampleCount"] = i
|
||||
}
|
||||
if f, ok := v.(float64); ok {
|
||||
body.Parameters["sampleCount"] = int(f)
|
||||
}
|
||||
}
|
||||
}
|
||||
if _, ok := body.Parameters["sampleCount"]; !ok {
|
||||
body.Parameters["sampleCount"] = 1
|
||||
}
|
||||
|
||||
if body.Parameters["sampleCount"].(int) <= 0 {
|
||||
return nil, fmt.Errorf("sampleCount must be greater than 0")
|
||||
}
|
||||
|
||||
// if req.Duration > 0 {
|
||||
// body.Parameters["durationSeconds"] = req.Duration
|
||||
// } else if req.Seconds != "" {
|
||||
// seconds, err := strconv.Atoi(req.Seconds)
|
||||
// if err != nil {
|
||||
// return nil, errors.Wrap(err, "convert seconds to int failed")
|
||||
// }
|
||||
// body.Parameters["durationSeconds"] = seconds
|
||||
// }
|
||||
|
||||
info.PriceData.OtherRatios = map[string]float64{
|
||||
"sampleCount": float64(body.Parameters["sampleCount"].(int)),
|
||||
}
|
||||
|
||||
// if v, ok := body.Parameters["durationSeconds"]; ok {
|
||||
// info.PriceData.OtherRatios["durationSeconds"] = float64(v.(int))
|
||||
// }
|
||||
|
||||
data, err := json.Marshal(body)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
|
||||
@@ -1,10 +1,7 @@
|
||||
package common
|
||||
|
||||
import (
|
||||
"encoding/base64"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -229,54 +226,3 @@ func ValidateBasicTaskRequest(c *gin.Context, info *RelayInfo, action string) *d
|
||||
storeTaskRequest(c, info, action, req)
|
||||
return nil
|
||||
}
|
||||
func GetImagesBase64sFromForm(c *gin.Context) ([]*Base64Data, error) {
|
||||
return GetBase64sFromForm(c, "image")
|
||||
}
|
||||
func GetImageBase64sFromForm(c *gin.Context) (*Base64Data, error) {
|
||||
base64s, err := GetImagesBase64sFromForm(c)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
return base64s[0], nil
|
||||
}
|
||||
|
||||
type Base64Data struct {
|
||||
MimeType string
|
||||
Data string
|
||||
}
|
||||
|
||||
func (m Base64Data) String() string {
|
||||
return fmt.Sprintf("data:%s;base64,%s", m.MimeType, m.Data)
|
||||
}
|
||||
func GetBase64sFromForm(c *gin.Context, fieldName string) ([]*Base64Data, error) {
|
||||
mf := c.Request.MultipartForm
|
||||
if mf == nil {
|
||||
if _, err := c.MultipartForm(); err != nil {
|
||||
return nil, fmt.Errorf("failed to parse image edit form request: %w", err)
|
||||
}
|
||||
mf = c.Request.MultipartForm
|
||||
}
|
||||
imageFiles, exists := mf.File[fieldName]
|
||||
if !exists || len(imageFiles) == 0 {
|
||||
return nil, errors.New("field " + fieldName + " is not found or empty")
|
||||
}
|
||||
var imageBase64s []*Base64Data
|
||||
for _, file := range imageFiles {
|
||||
image, err := file.Open()
|
||||
if err != nil {
|
||||
return nil, errors.New("failed to open image file")
|
||||
}
|
||||
defer image.Close()
|
||||
imageData, err := io.ReadAll(image)
|
||||
if err != nil {
|
||||
return nil, errors.New("failed to read image file")
|
||||
}
|
||||
mimeType := http.DetectContentType(imageData)
|
||||
base64Data := base64.StdEncoding.EncodeToString(imageData)
|
||||
imageBase64s = append(imageBase64s, &Base64Data{
|
||||
MimeType: mimeType,
|
||||
Data: base64Data,
|
||||
})
|
||||
}
|
||||
return imageBase64s, nil
|
||||
}
|
||||
|
||||
@@ -141,7 +141,6 @@ func GetAndValidOpenAIImageRequest(c *gin.Context, relayMode int) (*dto.ImageReq
|
||||
imageRequest.N = uint(common.String2Int(formData.Get("n")))
|
||||
imageRequest.Quality = formData.Get("quality")
|
||||
imageRequest.Size = formData.Get("size")
|
||||
imageRequest.ResponseFormat = formData.Get("response_format")
|
||||
if imageValue := formData.Get("image"); imageValue != "" {
|
||||
imageRequest.Image, _ = json.Marshal(imageValue)
|
||||
}
|
||||
|
||||
@@ -26,10 +26,6 @@ var defaultGeminiSettings = GeminiSettings{
|
||||
SupportedImagineModels: []string{
|
||||
"gemini-2.0-flash-exp-image-generation",
|
||||
"gemini-2.0-flash-exp",
|
||||
"gemini-3-pro-image-preview",
|
||||
"gemini-2.5-flash-image",
|
||||
"nano-banana",
|
||||
"nano-banana-pro",
|
||||
},
|
||||
ThinkingAdapterEnabled: false,
|
||||
ThinkingAdapterBudgetTokensPercentage: 0.6,
|
||||
|
||||
Reference in New Issue
Block a user