github/new-api
1
0
Fork 0
mirror of https://github.com/QuantumNous/new-api.git synced 2026-04-18 09:57:27 +00:00
Code Issues Packages Projects Releases Wiki Activity

✨ feat(models-sync): official upstream sync with conflict resolution UI, opt‑out flag, and backend resiliency

Browse Source 
Backend
- Add endpoints:
  - GET /api/models/sync_upstream/preview — diff preview (filters out models with sync_official = 0)
  - POST /api/models/sync_upstream — apply sync (create missing; optionally overwrite selected fields)
- Respect opt‑out: skip models with sync_official = 0 in both preview and apply
- Return detailed stats: created_models, created_vendors, updated_models, skipped_models, plus created_list / updated_list
- Add model.Model.SyncOfficial (default 1); auto‑migrated by GORM
- Make HTTP fetching robust:
  - Shared http.Client (connection reuse) with 3x exponential backoff retry
  - 10MB response cap; keep existing IPv4‑first for *.github.io
- Vendor handling:
  - New ensureVendorID helper (cache lookup → DB lookup → create), reduces round‑trips
  - Transactional overwrite to avoid partial updates
- Small cleanups and clearer helpers (containsField, coalesce, chooseStatus)

Frontend
- ModelsActions: add “Sync official” button with Popover (p‑2) explaining community contribution; loading = syncing || previewing; preview → conflict modal → apply flow
- New UpstreamConflictModal:
  - Per‑field columns (description/icon/tags/vendor/name_rule/status) with column‑level checkbox to select all
  - Cell with Checkbox + Tag (“Click to view differences”) and Popover (p‑2) showing Local vs Official values
  - Auto‑hide columns with no conflicts; responsive width; use native Semi Modal footer
  - Full i18n coverage
- useModelsData: add syncing/previewing states; new methods previewUpstreamDiff, applyUpstreamOverwrite, syncUpstream; refresh vendors/models after apply
- EditModelModal: add “Participate in official sync” switch; persisted as sync_official
- ModelsColumnDefs: add “Participate in official sync” column

i18n
- Add missing English keys for the new UI and messages; fix quoting issues

Refs
- Upstream metadata: https://github.com/basellm/llm-metadata
This commit is contained in:
t0ng7u
2025-09-02 02:04:22 +08:00
parent 1f111a163a
commit fbc19abd28
10 changed files with 902 additions and 22 deletions
Download Patch File Download Diff File Expand all files Collapse all files

66
web/src/components/table/models/ModelsActions.jsx
View File

@@ -21,10 +21,11 @@ import React, { useState } from 'react';
import MissingModelsModal from './modals/MissingModelsModal';
import PrefillGroupManagement from './modals/PrefillGroupManagement';
import EditPrefillGroupModal from './modals/EditPrefillGroupModal';
import { Button, Modal } from '@douyinfe/semi-ui';
import { Button, Modal, Popover } from '@douyinfe/semi-ui';
import { showSuccess, showError, copy } from '../../../helpers';
import CompactModeToggle from '../../common/ui/CompactModeToggle';
import SelectionNotification from './components/SelectionNotification';
import UpstreamConflictModal from './modals/UpstreamConflictModal';
const ModelsActions = ({
selectedKeys,
@@ -32,6 +33,11 @@ const ModelsActions = ({
setEditingModel,
setShowEdit,
batchDeleteModels,
syncing,
previewing,
syncUpstream,
previewUpstreamDiff,
applyUpstreamOverwrite,
compactMode,
setCompactMode,
t,
@@ -42,6 +48,21 @@ const ModelsActions = ({
const [showGroupManagement, setShowGroupManagement] = useState(false);
const [showAddPrefill, setShowAddPrefill] = useState(false);
const [prefillInit, setPrefillInit] = useState({ id: undefined });
const [showConflict, setShowConflict] = useState(false);
const [conflicts, setConflicts] = useState([]);
const handleSyncUpstream = async () => {
// 先预览
const data = await previewUpstreamDiff?.();
const conflictItems = data?.conflicts || [];
if (conflictItems.length > 0) {
setConflicts(conflictItems);
setShowConflict(true);
return;
}
// 无冲突，直接同步缺失
await syncUpstream?.();
};
// Handle delete selected models with confirmation
const handleDeleteSelectedModels = () => {
@@ -104,6 +125,38 @@ const ModelsActions = ({
{t('未配置模型')}
</Button>
<Popover
position='bottom'
trigger='hover'
content={
<div className='p-2 max-w-[360px]'>
<div className='text-[var(--semi-color-text-2)] text-sm'>
{t(
'模型社区需要大家的共同维护，如发现数据有误或想贡献新的模型数据，请访问：',
)}
</div>
<a
href='https://github.com/basellm/llm-metadata'
target='_blank'
rel='noreferrer'
className='text-blue-600 underline'
>
https://github.com/basellm/llm-metadata
</a>
</div>
}
>
<Button
type='secondary'
className='flex-1 md:flex-initial'
size='small'
loading={syncing || previewing}
onClick={handleSyncUpstream}
>
{t('同步官方')}
</Button>
</Popover>
<Button
type='secondary'
className='flex-1 md:flex-initial'
@@ -165,6 +218,17 @@ const ModelsActions = ({
editingGroup={prefillInit}
onSuccess={() => setShowAddPrefill(false)}
/>
<UpstreamConflictModal
visible={showConflict}
onClose={() => setShowConflict(false)}
conflicts={conflicts}
onSubmit={async (payload) => {
return await applyUpstreamOverwrite?.(payload);
}}
t={t}
loading={syncing}
/>
</>
);
};

9
web/src/components/table/models/ModelsColumnDefs.jsx
View File

@@ -303,6 +303,15 @@ export const getModelsColumns = ({
dataIndex: 'name_rule',
render: (val, record) => renderNameRule(val, record, t),
},
{
title: t('参与官方同步'),
dataIndex: 'sync_official',
render: (val) => (
<Tag size='small' shape='circle' color={val === 1 ? 'green' : 'orange'}>
{val === 1 ? t('是') : t('否')}
</Tag>
),
},
{
title: t('描述'),
dataIndex: 'description',

5
web/src/components/table/models/index.jsx
View File

@@ -105,6 +105,11 @@ const ModelsPage = () => {
setEditingModel={setEditingModel}
setShowEdit={setShowEdit}
batchDeleteModels={batchDeleteModels}
syncing={modelsData.syncing}
syncUpstream={modelsData.syncUpstream}
previewing={modelsData.previewing}
previewUpstreamDiff={modelsData.previewUpstreamDiff}
applyUpstreamOverwrite={modelsData.applyUpstreamOverwrite}
compactMode={compactMode}
setCompactMode={setCompactMode}
t={t}

15
web/src/components/table/models/modals/EditModelModal.jsx
View File

@@ -121,6 +121,7 @@ const EditModelModal = (props) => {
endpoints: '',
name_rule: props.editingModel?.model_name ? 0 : undefined, // 通过未配置模型过来的固定为精确匹配
status: true,
sync_official: true,
});
const handleCancel = () => {
@@ -145,8 +146,9 @@ const EditModelModal = (props) => {
if (!data.endpoints) {
data.endpoints = '';
}
// 处理status，将数字转为布尔值
// 处理status/sync_official，将数字转为布尔值
data.status = data.status === 1;
data.sync_official = (data.sync_official ?? 1) === 1;
if (formApiRef.current) {
formApiRef.current.setValues({ ...getInitValues(), ...data });
}
@@ -193,6 +195,7 @@ const EditModelModal = (props) => {
tags: Array.isArray(values.tags) ? values.tags.join(',') : values.tags,
endpoints: values.endpoints || '',
status: values.status ? 1 : 0,
sync_official: values.sync_official ? 1 : 0,
};
if (isEdit) {
@@ -505,6 +508,16 @@ const EditModelModal = (props) => {
}
/>
</Col>
<Col span={24}>
<Form.Switch
field='sync_official'
label={t('参与官方同步')}
extraText={t(
'关闭后，此模型将不会被“同步官方”自动覆盖或创建',
)}
size='large'
/>
</Col>
<Col span={24}>
<Form.Switch
field='status'

245
web/src/components/table/models/modals/UpstreamConflictModal.jsx Normal file
View File

@@ -0,0 +1,245 @@
/*
Copyright (C) 2025 QuantumNous
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as
published by the Free Software Foundation, either version 3 of the
License, or (at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
For commercial licensing, please contact support@quantumnous.com
*/
import React, { useEffect, useMemo, useState } from 'react';
import {
Modal,
Table,
Checkbox,
Typography,
Empty,
Tag,
Popover,
} from '@douyinfe/semi-ui';
import { MousePointerClick } from 'lucide-react';
import { useIsMobile } from '../../../../hooks/common/useIsMobile';
const { Text } = Typography;
const FIELD_LABELS = {
description: '描述',
icon: '图标',
tags: '标签',
vendor: '供应商',
name_rule: '命名规则',
status: '状态',
};
const FIELD_KEYS = Object.keys(FIELD_LABELS);
const UpstreamConflictModal = ({
visible,
onClose,
conflicts = [],
onSubmit,
t,
loading = false,
}) => {
const [selections, setSelections] = useState({});
const isMobile = useIsMobile();
const formatValue = (v) => {
if (v === null || v === undefined) return '-';
if (typeof v === 'string') return v || '-';
try {
return JSON.stringify(v, null, 2);
} catch (_) {
return String(v);
}
};
useEffect(() => {
if (visible) {
const init = {};
conflicts.forEach((item) => {
init[item.model_name] = new Set();
});
setSelections(init);
} else {
setSelections({});
}
}, [visible, conflicts]);
const toggleField = (modelName, field, checked) => {
setSelections((prev) => {
const next = { ...prev };
const set = new Set(next[modelName] || []);
if (checked) set.add(field);
else set.delete(field);
next[modelName] = set;
return next;
});
};
const columns = useMemo(() => {
const base = [
{
title: t('模型'),
dataIndex: 'model_name',
render: (text) => <Text strong>{text}</Text>,
},
];
const cols = FIELD_KEYS.map((fieldKey) => {
const rawLabel = FIELD_LABELS[fieldKey] || fieldKey;
const label = t(rawLabel);
// 统计列头复选框状态（仅统计存在该字段冲突的行）
const presentRows = (conflicts || []).filter((row) =>
(row.fields || []).some((f) => f.field === fieldKey),
);
const selectedCount = presentRows.filter((row) =>
selections[row.model_name]?.has(fieldKey),
).length;
const allCount = presentRows.length;
if (allCount === 0) {
return null; // 若此字段在所有行中都不存在，则不展示该列
}
const headerChecked = allCount > 0 && selectedCount === allCount;
const headerIndeterminate = selectedCount > 0 && selectedCount < allCount;
const onHeaderChange = (e) => {
const checked = e?.target?.checked;
setSelections((prev) => {
const next = { ...prev };
(conflicts || []).forEach((row) => {
const hasField = (row.fields || []).some(
(f) => f.field === fieldKey,
);
if (!hasField) return;
const set = new Set(next[row.model_name] || []);
if (checked) set.add(fieldKey);
else set.delete(fieldKey);
next[row.model_name] = set;
});
return next;
});
};
return {
title: (
<div className='flex items-center gap-2'>
<Checkbox
checked={headerChecked}
indeterminate={headerIndeterminate}
onChange={onHeaderChange}
/>
<Text>{label}</Text>
</div>
),
dataIndex: fieldKey,
render: (_, record) => {
const f = (record.fields || []).find((x) => x.field === fieldKey);
if (!f) return <Text type='tertiary'>-</Text>;
const checked = selections[record.model_name]?.has(fieldKey) || false;
return (
<Checkbox
checked={checked}
onChange={(e) =>
toggleField(record.model_name, fieldKey, e?.target?.checked)
}
>
<Popover
trigger='hover'
position='top'
content={
<div className='p-2 max-w-[520px]'>
<div className='mb-2'>
<Text type='tertiary' size='small'>
{t('本地')}
</Text>
<pre className='whitespace-pre-wrap m-0'>
{formatValue(f.local)}
</pre>
</div>
<div>
<Text type='tertiary' size='small'>
{t('官方')}
</Text>
<pre className='whitespace-pre-wrap m-0'>
{formatValue(f.upstream)}
</pre>
</div>
</div>
}
>
<Tag
color='white'
size='small'
prefixIcon={<MousePointerClick size={14} />}
>
{t('点击查看差异')}
</Tag>
</Popover>
</Checkbox>
);
},
};
});
return [...base, ...cols.filter(Boolean)];
}, [t, selections, conflicts]);
const dataSource = conflicts.map((c) => ({
key: c.model_name,
model_name: c.model_name,
fields: c.fields || [],
}));
const handleOk = async () => {
const payload = Object.entries(selections)
.map(([modelName, set]) => ({
model_name: modelName,
fields: Array.from(set || []),
}))
.filter((x) => x.fields.length > 0);
if (payload.length === 0) {
onClose?.();
return;
}
const ok = await onSubmit?.(payload);
if (ok) onClose?.();
};
return (
<Modal
title={t('选择要覆盖的冲突项')}
visible={visible}
onCancel={onClose}
onOk={handleOk}
confirmLoading={loading}
okText={t('应用覆盖')}
cancelText={t('取消')}
width={isMobile ? '100%' : 1000}
>
{dataSource.length === 0 ? (
<Empty description={t('无冲突项')} className='p-6' />
) : (
<>
<div className='mb-3 text-[var(--semi-color-text-2)]'>
{t('仅会覆盖你勾选的字段，未勾选的字段保持本地不变。')}
</div>
<Table columns={columns} dataSource={dataSource} pagination={false} />
</>
)}
</Modal>
);
};
export default UpstreamConflictModal;

84
web/src/hooks/models/useModelsData.jsx
View File

@@ -95,6 +95,8 @@ export const useModelsData = () => {
const [showAddVendor, setShowAddVendor] = useState(false);
const [showEditVendor, setShowEditVendor] = useState(false);
const [editingVendor, setEditingVendor] = useState({ id: undefined });
const [syncing, setSyncing] = useState(false);
const [previewing, setPreviewing] = useState(false);
const vendorMap = useMemo(() => {
const map = {};
@@ -163,6 +165,81 @@ export const useModelsData = () => {
await loadModels(page, pageSize);
};
// Sync upstream models/vendors for missing models only
const syncUpstream = async () => {
setSyncing(true);
try {
const res = await API.post('/api/models/sync_upstream');
const { success, message, data } = res.data || {};
if (success) {
const createdModels = data?.created_models || 0;
const createdVendors = data?.created_vendors || 0;
const skipped = (data?.skipped_models || []).length || 0;
showSuccess(
t(
`已同步：新增 ${createdModels} 模型，新增 ${createdVendors} 供应商，跳过 ${skipped} 项`,
),
);
await loadVendors();
await refresh();
} else {
showError(message || t('同步失败'));
}
} catch (e) {
showError(t('同步失败'));
}
setSyncing(false);
};
// Preview upstream differences
const previewUpstreamDiff = async () => {
setPreviewing(true);
try {
const res = await API.get('/api/models/sync_upstream/preview');
const { success, message, data } = res.data || {};
if (success) {
return data || { missing: [], conflicts: [] };
}
showError(message || t('预览失败'));
return { missing: [], conflicts: [] };
} catch (e) {
showError(t('预览失败'));
return { missing: [], conflicts: [] };
} finally {
setPreviewing(false);
}
};
// Apply selected overwrite
const applyUpstreamOverwrite = async (overwrite = []) => {
setSyncing(true);
try {
const res = await API.post('/api/models/sync_upstream', { overwrite });
const { success, message, data } = res.data || {};
if (success) {
const createdModels = data?.created_models || 0;
const updatedModels = data?.updated_models || 0;
const createdVendors = data?.created_vendors || 0;
const skipped = (data?.skipped_models || []).length || 0;
showSuccess(
t(
`完成：新增 ${createdModels} 模型，更新 ${updatedModels} 模型，新增 ${createdVendors} 供应商，跳过 ${skipped} 项`,
),
);
await loadVendors();
await refresh();
return true;
}
showError(message || t('同步失败'));
return false;
} catch (e) {
showError(t('同步失败'));
return false;
} finally {
setSyncing(false);
}
};
// Search models with keyword and vendor
const searchModels = async () => {
const { searchKeyword = '', searchVendor = '' } = getFormValues();
@@ -398,5 +475,12 @@ export const useModelsData = () => {
// Translation
t,
// Upstream sync
syncing,
previewing,
syncUpstream,
previewUpstreamDiff,
applyUpstreamOverwrite,
};
};

34
web/src/i18n/locales/en.json
View File

@@ -1,6 +1,5 @@
{
"主页": "Home",
"文档": "Docs",
"控制台": "Console",
"$%.6f 额度": "$%.6f quota",
"或": "or",
@@ -165,7 +164,6 @@
"出现错误，第 ${count} 次重试中...": "Error occurred, retry attempt ${count}...",
"首页": "Home",
"渠道": "Channel",
"渠道管理": "Channels",
"令牌": "Tokens",
"兑换额度": "Redeem",
"充值": "Recharge",
@@ -174,7 +172,6 @@
"设置": "Settings",
"关于": "About",
"价格": "Pricing",
"聊天": "Chat",
"注销成功!": "Logout successful!",
"注销": "Logout",
"登录": "Sign in",
@@ -443,9 +440,7 @@
"兑换码": "Redeem Code",
"管理用户": "Manage Users",
"额度明细": "Quota Details",
"个人设置": "Personal Settings",
"运营设置": "Operation Settings",
"系统设置": "System Settings",
"其他设置": "Other Settings",
"项目仓库地址": "Project Repository Address",
"可在设置页面设置关于内容，支持 HTML & Markdown": "The About content can be set on the settings page, supporting HTML & Markdown",
@@ -585,10 +580,7 @@
"确定是否要修复数据库一致性？": "Are you sure you want to repair database consistency?",
"进行该操作时，可能导致渠道访问错误，请仅在数据库出现问题时使用": "When performing this operation, it may cause channel access errors. Please only use it when there is a problem with the database.",
"当前没有可用的启用令牌，请确认是否有令牌处于启用状态！": "There are currently no enablement tokens available, please confirm if one is enabled!",
"令牌管理": "API Keys",
"使用日志": "Usage log",
"Midjourney日志": "Midjourney",
"数据看板": "Dashboard",
"模型列表": "Model list",
"常见问题": "FAQ",
"免费体验": "Free trial",
@@ -628,7 +620,6 @@
"重置成功": "Reset successful",
"加载数据出错:": "Error loading data:",
"加载数据时发生错误: ": "An error occurred while loading data:",
"保存成功": "Saved successfully",
"部分保存失败，请重试": "Partial saving failed, please try again",
"请检查输入": "Please check your input",
"如何区分不同分组不同模型的价格：供参考的配置方式": "How to distinguish the prices of different models in different groups: configuration method for reference",
@@ -653,7 +644,6 @@
"窗口等待": "window wait",
"失败": "Failed",
"绘图": "Drawing",
"绘图日志": "Drawing log",
"放大": "Upscalers",
"微妙放大": "Upscale (Subtle)",
"创造放大": "Upscale (Creative)",
@@ -793,7 +783,6 @@
"邮箱": "Email",
"已有账户？": "Already have an account?",
"创意任务": "Tasks",
"用户管理": "User Management",
"任务ID（点击查看详情）": "Task ID (click to view details)",
"进度": "schedule",
"花费时间": "spend time",
@@ -943,7 +932,6 @@
"不是合法的 JSON 字符串": "Not a valid JSON string",
"个人中心": "Personal center",
"代理商": "Agent",
"钱包管理": "Wallet",
"备注": "Remark",
"工作台": "Workbench",
"已复制：": "Copied:",
@@ -957,7 +945,6 @@
"黑夜模式": "Dark mode",
"管理员设置": "Admin",
"待更新": "To be updated",
"模型广场": "Pricing",
"支付中..": "Paying",
"查看图片": "View pictures",
"并发限制": "Concurrency limit",
@@ -1043,7 +1030,6 @@
"在iframe中加载": "Load in iframe",
"补全倍率": "Completion ratio",
"保存分组数据失败": "Failed to save group data",
"保存失败，请重试": "Save failed, please try again",
"没有可用的使用信息": "No usage information available",
"使用详情": "Usage details",
"收起": "Collapse",
@@ -1187,7 +1173,6 @@
"知识库 ID": "Knowledge Base ID",
"请输入知识库 ID，例如：123456": "Please enter knowledge base ID, e.g.: 123456",
"可选值": "Optional value",
"任务日志": "Task log",
"你好": "Hello",
"你好，请问有什么可以帮助您的吗？": "Hello, how may I help you?",
"用户分组": "Your default group",
@@ -1329,7 +1314,6 @@
"当剩余额度低于此数值时，系统将通过选择的方式发送通知": "When the remaining quota is lower than this value, the system will send a notification through the selected method",
"Webhook请求结构": "Webhook request structure",
"只支持https，系统将以 POST 方式发送通知，请确保地址可以接收 POST 请求": "Only https is supported, the system will send a notification through POST, please ensure the address can receive POST requests",
"保存设置": "Save settings",
"通知邮箱": "Notification email",
"设置用于接收额度预警的邮箱地址，不填则使用账号绑定的邮箱": "Set the email address for receiving quota warning notifications, if not set, the email address bound to the account will be used",
"留空则使用账号绑定的邮箱": "If left blank, the email address bound to the account will be used",
@@ -1501,7 +1485,6 @@
"收益": "Earnings",
"无邀请人": "No Inviter",
"邀请人": "Inviter",
"兑换码管理": "Redemption Code",
"设置兑换码的基本信息": "Set redemption code basic information",
"设置兑换码的额度和数量": "Set redemption code quota and quantity",
"编辑用户": "Edit User",
@@ -1595,7 +1578,6 @@
"加载中...": "Loading...",
"正在跳转...": "Redirecting...",
"暂无公告": "No Notice",
"操练场": "Playground",
"欢迎使用，请完成以下设置以开始使用系统": "Welcome to use, please complete the following settings to start using the system",
"数据库检查": "Database Check",
"验证数据库连接状态": "Verify database connection status",
@@ -1811,7 +1793,6 @@
"系统提示覆盖": "System prompt override",
"模型: {{ratio}}": "Model: {{ratio}}",
"专属倍率": "Exclusive group ratio",
"模型管理": "Models",
"匹配类型": "Matching type",
"描述": "Description",
"供应商": "Vendor",
@@ -2075,5 +2056,18 @@
"保存边栏设置": "Save Sidebar Settings",
"侧边栏设置保存成功": "Sidebar settings saved successfully",
"需要登录访问": "Require Login",
"开启后未登录用户无法访问模型广场": "When enabled, unauthenticated users cannot access the model marketplace"
"开启后未登录用户无法访问模型广场": "When enabled, unauthenticated users cannot access the model marketplace",
"同步官方": "Sync official",
"参与官方同步": "Participate in official sync",
"关闭后，此模型将不会被“同步官方”自动覆盖或创建": "When turned off, this model will be skipped by Sync official (no auto create/overwrite)",
"选择要覆盖的冲突项": "Select conflict items to overwrite",
"点击查看差异": "Click to view differences",
"无冲突项": "No conflict items",
"应用覆盖": "Apply overwrite",
"仅会覆盖你勾选的字段，未勾选的字段保持本地不变。": "Only selected fields will be overwritten; unselected fields remain unchanged.",
"本地": "Local",
"官方": "Official",
"模型社区需要大家的共同维护，如发现数据有误或想贡献新的模型数据，请访问：": "The model community needs everyone's contribution. If you find incorrect data or want to contribute new models, please visit:",
"是": "Yes",
"否": "No"
}