Frontend (web)
- ModelsActions.jsx
- Replace “Sync Official” with “Sync” and open a new two-step SyncWizard.
- Pass selected locale through to preview, sync, and overwrite flows.
- Keep conflict resolution flow; inject locale into overwrite submission.
- New: models/modals/SyncWizardModal.jsx
- Two-step wizard: (1) method selection (config-sync disabled for now), (2) language selection (en/zh/ja).
- Horizontal, centered Radio cards; returns { option, locale } via onConfirm.
- UpstreamConflictModal.jsx
- Add search input (model fuzzy search) and native pagination.
- Column header checkbox now only applies to rows in the current filtered result.
- Fix “Cannot access ‘filteredDataSource’ before initialization”.
- Refactor with useMemo/useCallback; extract helpers to remove duplicated logic:
- getPresentRowsForField, getHeaderState, applyHeaderChange
- Minor code cleanups and stability improvements.
- i18n (en.json)
- Add strings for the sync wizard and related actions (Sync, Sync Wizard, Select method/source/language, etc.).
- Adjust minor translations.
Hooks
- useModelsData.jsx
- Extend previewUpstreamDiff, syncUpstream, applyUpstreamOverwrite to accept options with locale.
- Send locale via query/body accordingly.
Backend (Go)
- controller/model_sync.go
- Accept locale from query/body and resolve i18n upstream URLs.
- Add SYNC_UPSTREAM_BASE for upstream base override (default: https://basellm.github.io/llm-metadata).
- Make HTTP timeouts/retries/limits configurable:
- SYNC_HTTP_TIMEOUT_SECONDS, SYNC_HTTP_RETRY, SYNC_HTTP_MAX_MB
- Add ETag-based caching and support both envelope and pure array JSON formats.
- Concurrently fetch vendors and models; improve error responses with locale and source URLs.
- Include source meta (locale, models_url, vendors_url) in success payloads.
Notes
- No breaking changes expected.
- Lint passes for touched files.
- Delete dropIndexIfExists helper from `model/main.go`
- Remove all calls to dropIndexIfExists in `migrateDB` and `migrateDBFast`
- Drop related comments and MySQL-only DROP INDEX code paths
- Keep GORM AutoMigrate as the sole migration path for `Model` and `Vendor`
Why:
- Simplifies migrations and avoids destructive index drops at startup
- Prevents noisy MySQL 1091 errors and vendor-specific branches
- Aligns with composite unique indexes (uk_model_name_delete_at, uk_vendor_name_delete_at)
Impact:
- No expected runtime behavior change; schema remains managed by GORM
- Legacy single-column unique indexes (if any) will no longer be auto-dropped
- Safe across MySQL/PostgreSQL/SQLite; MySQL Chinese charset checks remain intact
Verification:
- Lint passed for `model/main.go`
- Confirmed no remaining `DROP INDEX` or `dropIndexIfExists` references
- Model: rename `uk_model_name` -> `uk_model_name_delete_at`
(composite on `model_name` + `deleted_at`)
- Vendor: rename `uk_vendor_name` -> `uk_vendor_name_delete_at`
(composite on `name` + `deleted_at`)
- Keep legacy cleanup in `model/main.go` to drop old index names
(`uk_model_name`, `model_name`, `uk_vendor_name`, `name`) for compatibility.
Result: idempotent GORM migrations and no unnecessary index churn on MySQL restarts.
Files:
- `model/model_meta.go`
- `model/vendor_meta.go`
- Add defaultVendorIcons mapping for major AI vendors
- Update getOrCreateVendor to automatically set vendor icons
- Add getDefaultVendorIcon helper function
- Support LobeHub icons Color variants (e.g., Claude.Color, Gemini.Color)
- Fix issue where default vendors were created without icons
This ensures that when new models are encountered, their vendors
will be created with appropriate colored icons for better UI display.
Affected vendors include:
- OpenAI, Anthropic, Google, Moonshot, 智谱, 阿里巴巴
- DeepSeek, MiniMax, 百度, 讯飞, 腾讯, Cohere
- Cloudflare, 360, 零一万物, Jina, Mistral, xAI
- Meta, 字节跳动, 快手, 即梦, Vidu, Microsoft/Azure
This commit refactors the logging mechanism across the application by replacing direct logger calls with a centralized logging approach using the `common` package. Key changes include:
- Replaced instances of `logger.SysLog` and `logger.FatalLog` with `common.SysLog` and `common.FatalLog` for consistent logging practices.
- Updated resource initialization error handling to utilize the new logging structure, enhancing maintainability and readability.
- Minor adjustments to improve code clarity and organization throughout various modules.
This change aims to streamline logging and improve the overall architecture of the codebase.
This commit introduces a major architectural refactoring to improve quota management, centralize logging, and streamline the relay handling logic.
Key changes:
- **Pre-consume Quota:** Implements a new mechanism to check and reserve user quota *before* making the request to the upstream provider. This ensures more accurate quota deduction and prevents users from exceeding their limits due to concurrent requests.
- **Unified Relay Handlers:** Refactors the relay logic to use generic handlers (e.g., `ChatHandler`, `ImageHandler`) instead of provider-specific implementations. This significantly reduces code duplication and simplifies adding new channels.
- **Centralized Logger:** A new dedicated `logger` package is introduced, and all system logging calls are migrated to use it, moving this responsibility out of the `common` package.
- **Code Reorganization:** DTOs are generalized (e.g., `dalle.go` -> `openai_image.go`) and utility code is moved to more appropriate packages (e.g., `common/http.go` -> `service/http.go`) for better code structure.
- Added a check for MySQL charset/collation to ensure compatibility with Chinese characters during database initialization.
- Updated SQLite busy timeout from 5000ms to 30000ms for improved performance.
- Removed commented-out PostgreSQL migration logic for clarity.
Backend
- Add GetBoundChannelsByModelsMap to batch-fetch bound channels via a single JOIN (Distinct), compatible with SQLite/MySQL/PostgreSQL
- Replace per-record enrichment with a single-pass enrichModels to avoid N+1 queries; compute unions for prefix/suffix/contains matches in memory
- Change Model.QuotaType to QuotaTypes []int and expose quota_types in responses
- Add GetModelQuotaTypes for cached O(1) lookups; exact models return a single-element array
- Sort quota_types for stable output order
- Remove unused code: GetModelByName, GetBoundChannels, GetBoundChannelsForModels, FindModelByNameWithRule, buildPrefixes, buildSuffixes
- Clean up redundant comments, keeping concise and readable code
Frontend
- Models table: switch to quota_types, render multiple billing modes ([0], [1], [0,1], future values supported)
- Pricing table: switch to quota_types; ratio display now checks quota_types.includes(0); array rendering for billing tags
Compatibility
- SQL uses standard JOIN/IN/Distinct; works across SQLite/MySQL/PostgreSQL
- Lint passes; no DB schema changes (quota_types is a JSON response field only)
Breaking Change
- API field renamed: quota_type -> quota_types (array). Update clients accordingly.
Why
Previous versions created single-column UNIQUE constraints (`models.model_name`, `vendors.name`).
After introducing composite indexes on `(model_name, deleted_at)` and `(name, deleted_at)` for soft-delete support, those legacy constraints could still exist in user databases.
When a record was soft-deleted and re-inserted with the same name, MySQL raised `Error 1062 … for key 'models.model_name'`.
What
• In `migrateDB` and `migrateDBFast` paths of `model/main.go`, proactively drop:
– `models.uk_model_name` and fallback `models.model_name`
– `vendors.uk_vendor_name` and fallback `vendors.name`
• Keeps existing helper `dropIndexIfExists` to ensure the operation is MySQL-only and error-free when indexes are already absent.
Result
Startup migration now removes every possible legacy UNIQUE index, ensuring composite index strategy works correctly.
Users can soft-delete and recreate models/vendors with identical names without hitting duplicate-entry errors.
Summary
-------
1. **Backend**
• `model/model_meta.go`
– Add `GetBoundChannelsForModels([]string)` to retrieve channels for multiple models in a single SQL (`IN (?)`) and deduplicate with `GROUP BY`.
• `controller/model_meta.go`
– In non-exact `fillModelExtra`:
– Remove per-model `GetBoundChannels` calls.
– Collect matched model names, then call `GetBoundChannelsForModels` once and merge results into `channelSet`.
– Minor cleanup on loop logic; channel aggregation now happens after quota/group/endpoint processing.
Impact
------
• Eliminates N+1 query pattern for prefix/suffix/contains rules.
• Reduces DB round-trips from *N + 1* to **1**, markedly speeding up the model-management list load.
• Keeps existing `GetBoundChannels` API intact for single-model scenarios; no breaking changes.
Summary
-------
1. **Backend**
• `model/model_meta.go`
– Add `MatchedModels []string` and `MatchedCount int` (ignored by GORM) to expose matching details in API responses.
• `controller/model_meta.go`
– When processing prefix/suffix/contains rules in `fillModelExtra`, collect every matched model name, fill `MatchedModels`, and calculate `MatchedCount`.
2. **Frontend**
• `web/src/components/table/models/ModelsColumnDefs.js`
– Import `Tooltip`.
– Enhance `renderNameRule` to:
– Display tag text like “前缀 5个模型” for non-exact rules.
– Show a tooltip listing all matched model names on hover.
Impact
------
Users now see the total number of concrete models aggregated under each prefix/suffix/contains rule and can inspect the exact list via tooltip, improving transparency in model management.
Backend:
- Model: Add `icon` field to `model.Model` (gorm: varchar(128)); auto-migrated via GORM.
- Pricing API: Extend `model.Pricing` with `icon` and populate from model meta in `GetPricing()`.
Frontend:
- EditModelModal: Add `icon` input (with @lobehub/icons helper link); wire into init/load/submit flows.
- ModelHeader / PricingCardView: Prefer rendering `model.icon`; fallback to `vendor_icon`; final fallback to initials avatar.
- Models table: Add leading “Icon” column, rendering `model.icon` or `vendor` icon via `getLobeHubIcon`.
Notes:
- Backward-compatible. Existing data without `icon` remain unaffected.
- No manual SQL needed; column is added by AutoMigrate.
Affected files:
- model/model_meta.go
- model/pricing.go
- web/src/components/table/models/modals/EditModelModal.jsx
- web/src/components/table/model-pricing/modal/components/ModelHeader.jsx
- web/src/components/table/model-pricing/view/card/PricingCardView.jsx
- web/src/components/table/models/ModelsColumnDefs.js
Replace legacy single-column unique indexes with composite unique indexes on
(name, deleted_at) and introduce a safe index drop utility to eliminate
duplicate-key errors and noisy MySQL 1091 warnings.
WHAT
• model/model_meta.go
- Model.ModelName → `uniqueIndex:uk_model_name,priority:1`
- Model.DeletedAt → `index; uniqueIndex:uk_model_name,priority:2`
• model/vendor_meta.go
- Vendor.Name → `uniqueIndex:uk_vendor_name,priority:1`
- Vendor.DeletedAt → `index; uniqueIndex:uk_vendor_name,priority:2`
• model/main.go
- Add `dropIndexIfExists(table, index)`:
• Checks `information_schema.statistics`
• Drops index only when present (avoids Error 1091)
- Invoke helper in `migrateDB` & `migrateDBFast`
- Remove direct `ALTER TABLE … DROP INDEX …` calls
WHY
• Users received `Error 1062 (23000)` when re-creating a soft-deleted
model/vendor because the old unique index enforced uniqueness on name alone.
• Directly dropping nonexistent indexes caused MySQL `Error 1091` noise.
HOW
• Composite unique indexes `(model_name, deleted_at)` / `(name, deleted_at)`
respect GORM soft deletes.
• Safe helper ensures idempotent migrations across environments.
RESULT
• Users can now delete and re-add the same model or vendor without manual SQL.
• Startup migration runs quietly across MySQL, PostgreSQL, and SQLite.
• No behavior changes for existing data beyond index updates.
TEST
1. Add model “deepseek-chat” → delete (soft) → re-add → success.
2. Add vendor “DeepSeek” → delete (soft) → re-add → success.
3. Restart service twice → no duplicate key or 1091 errors.
Ensure models and vendors can be re-created after soft deletion by switching to composite unique indexes on (name, deleted_at) and cleaning up legacy single-column unique indexes on MySQL.
Why
- MySQL raised 1062 duplicate key errors when re-adding a soft-deleted model/vendor because the legacy unique index enforced uniqueness on the name column alone (uk_model_name / uk_vendor_name), despite soft deletes.
- Users encountered errors such as:
- Error 1062 (23000): Duplicate entry 'deepseek-chat' for key 'models.uk_model_name'
- Error 1062 (23000): Duplicate entry 'DeepSeek' for key 'vendors.uk_vendor_name'
How
- Model indices:
- model/model_meta.go:
- Model.ModelName → gorm: uniqueIndex:uk_model_name,priority:1
- Model.DeletedAt → gorm: index; uniqueIndex:uk_model_name,priority:2
- Vendor indices:
- model/vendor_meta.go:
- Vendor.Name → gorm: uniqueIndex:uk_vendor_name,priority:1
- Vendor.DeletedAt → gorm: index; uniqueIndex:uk_vendor_name,priority:2
- Migration (automatic, idempotent):
- model/main.go (migrateDB, migrateDBFast):
- On MySQL, drop legacy single-column unique indexes if present:
- ALTER TABLE models DROP INDEX uk_model_name;
- ALTER TABLE vendors DROP INDEX uk_vendor_name;
- Then run AutoMigrate to create composite unique indexes.
- Missing-index errors are ignored to keep the migration safe to run multiple times.
Result
- Users can delete and re-add the same model/vendor name without manual SQL.
- Migration runs automatically at startup; no user action required.
- PostgreSQL and SQLite remain unaffected.
Files changed
- model/model_meta.go
- model/vendor_meta.go
- model/main.go (migrateDB, migrateDBFast)
Testing
- Create model "deepseek-chat" → delete (soft) → re-create → succeeds.
- Create vendor "DeepSeek" → delete (soft) → re-create → succeeds.
Backward compatibility
- Data remains intact; only index definitions are updated.
- Behavior is unchanged except for fixing the uniqueness constraint with soft deletes.
- Why:
- Avoid introducing `gorm.io/datatypes` for a single field.
- Align with existing pattern (`ChannelInfo`, `Properties`) using `Scanner`/`Valuer`.
- Fix runtime error when drivers return JSON as string.
- What:
- Introduced `JSONValue` (based on `json.RawMessage`) implementing `sql.Scanner` and `driver.Valuer`, with `MarshalJSON`/`UnmarshalJSON` to preserve raw JSON in API.
- Updated `PrefillGroup.Items` to use `JSONValue` with `gorm:"type:json"`.
- Localized comments in `model/prefill_group.go` to Chinese.
- Impact:
- Resolves “unsupported Scan, storing driver.Value type string into type *json.RawMessage”.
- Works with MySQL/Postgres/SQLite whether JSON is returned as `[]byte` or `string`.
- API and DB schema remain unchanged; no `go.mod` changes; lints pass.
Files changed:
- model/prefill_group.go
