513 Commits

Author SHA1 Message Date
CaIon
80c213072c fix: improve multipart form data handling in gin context
- Added caching for the original Content-Type header in the parseMultipartFormData function.
- This change ensures that the Content-Type is retrieved from the context if previously set, enhancing performance and consistency.
2026-02-23 16:59:46 +08:00
CaIon
13ada6484a feat(task): introduce task timeout configuration and cleanup unfinished tasks
- Added TaskTimeoutMinutes constant to configure the timeout duration for asynchronous tasks.
- Implemented sweepTimedOutTasks function to identify and handle unfinished tasks that exceed the timeout limit, marking them as failed and processing refunds if applicable.
- Enhanced task polling loop to include the new timeout handling logic, ensuring timely cleanup of stale tasks.
2026-02-22 17:59:38 +08:00
CaIon
ec5c6b28ea feat(task): add model redirection, per-call billing, and multipart retry fix for async tasks
1. Async task model redirection (aligned with sync tasks):
   - Integrate ModelMappedHelper in RelayTaskSubmit after model name
     determination, populating OriginModelName / UpstreamModelName on RelayInfo.
   - All task adaptors now send UpstreamModelName to upstream providers:
     - Gemini & Vertex: BuildRequestURL uses UpstreamModelName.
     - Doubao & Ali: BuildRequestBody conditionally overwrites body.Model.
     - Vidu, Kling, Hailuo, Jimeng: convertToRequestPayload accepts RelayInfo
       and unconditionally uses info.UpstreamModelName.
     - Sora: BuildRequestBody parses JSON and multipart bodies to replace
       the "model" field with UpstreamModelName.
   - Frontend log visibility: LogTaskConsumption and taskBillingOther now
     emit is_model_mapped / upstream_model_name in the "other" JSON field.
   - Billing safety: RecalculateTaskQuotaByTokens reads model name from
     BillingContext.OriginModelName (via taskModelName) instead of
     task.Data["model"], preventing billing leaks from upstream model names.

2. Per-call billing (TaskPricePatches lifecycle):
   - Rename TaskBillingContext.ModelName → OriginModelName; add PerCallBilling
     bool field, populated from TaskPricePatches at submission time.
   - settleTaskBillingOnComplete short-circuits when PerCallBilling is true,
     skipping both adaptor adjustments and token-based recalculation.
   - Remove ModelName from TaskSubmitResult; use relayInfo.OriginModelName
     consistently in controller/relay.go for billing context and logging.

3. Multipart retry boundary mismatch fix:
   - Root cause: after Sora (or OpenAI audio) rebuilds a multipart body with a
     new boundary and overwrites c.Request.Header["Content-Type"], subsequent
     calls to ParseMultipartFormReusable on retry would parse the cached
     original body with the wrong boundary, causing "NextPart: EOF".
   - Fix: ParseMultipartFormReusable now caches the original Content-Type in
     gin context key "_original_multipart_ct" on first call and reuses it for
     all subsequent parses, making multipart parsing retry-safe globally.
   - Sora adaptor reverted to the standard pattern (direct header set/get),
     which is now safe thanks to the root fix.

4. Tests:
   - task_billing_test.go: update makeTask to use OriginModelName; add
     PerCallBilling settlement tests (skip adaptor adjust, skip token recalc);
     add non-per-call adaptor adjustment test with refund verification.
2026-02-22 16:33:00 +08:00
CaIon
197b89ea58 feat: refactor request body handling to use BodyStorage for improved efficiency 2026-02-12 01:51:27 +08:00
funkpopo
75e533edb0 feat(xai): 为xAI渠道添加/v1/responses支持 (#2897)
* feat(xai): 为xAI渠道添加/v1/responses支持

* Add video generation model to constants

* fix: 修正先前更改中对于grok-3-mini的思考预算和"-search"设计
2026-02-12 00:42:39 +08:00
CaIon
44c5fac5ea refactor(ratio): replace maps with RWMap for improved concurrency handling 2026-02-08 00:48:21 +08:00
CaIon
d814d62e2f refactor: enhance API security with read-only token authentication and improved rate limiting 2026-02-06 21:26:26 +08:00
Calcium-Ion
053699fa98 Merge commit from fork
fix: harden token search with pagination, rate limiting and input validation
2026-02-06 17:54:40 +08:00
CaIon
3e1be18310 fix: harden token search with pagination, rate limiting and input validation
- Add configurable per-user token creation limit (max_user_tokens)
- Sanitize search input patterns to prevent expensive queries
- Add per-user search rate limiting (by user ID)
- Add pagination to search endpoint with strict page size cap
- Skip empty search fields instead of matching nothing
- Hide internal errors from API responses
- Fix Interface2String float64 formatting causing config parse failures
- Add float-string fallback in config system for int/uint fields
2026-02-06 17:52:19 +08:00
CaIon
f60fce6584 feat(i18n): add backend multi-language support with user language preference
- Add go-i18n library for internationalization
- Create i18n package with translation keys and YAML locale files (zh/en)
- Implement i18n middleware for language detection from user settings and Accept-Language header
- Add Language field to UserSetting DTO
- Update API response helpers with i18n support (ApiErrorI18n, ApiSuccessI18n)
- Migrate hardcoded messages in token, redemption, and user controllers
- Add frontend language preference settings component
- Sync language preference across header selector and user settings
- Auto-restore user language preference on login
2026-02-05 00:09:32 +08:00
CaIon
7a4fc68bcc feat(performance): implement system performance monitoring with configurable thresholds 2026-02-04 21:26:07 +08:00
CaIon
7cfed0df8e refactor(gemini): remove GeminiVisionMaxImageNum constant and related image count logic 2026-02-04 19:10:06 +08:00
CaIon
e2ebd42a8c feat(cache): enhance disk cache management with concurrency control and cleanup optimizations 2026-02-04 18:23:17 +08:00
CaIon
9ef7740fe7 feat(file): unify file handling with a new FileSource abstraction for URL and base64 data 2026-02-04 18:23:17 +08:00
同語
9da3412fde feat: add subscription billing system (#2808)
* ci: create docker automation

*  feat: add subscription billing system with admin management and user purchase flow

Implement a new subscription-based billing model alongside existing metered/per-request billing:

Backend:
- Add subscription plan models (SubscriptionPlan, SubscriptionPlanItem, UserSubscription, etc.)
- Implement CRUD APIs for subscription plan management (admin only)
- Add user subscription queries with support for multiple active/expired subscriptions
- Integrate payment gateways (Stripe, Creem, Epay) for subscription purchases
- Implement pre-consume and post-consume billing logic for subscription quota tracking
- Add billing preference settings (subscription_first, wallet_first, etc.)
- Enhance usage logs with subscription deduction details

Frontend - Admin:
- Add subscription management page with table view and drawer-based edit form
- Match UI/UX style with existing admin pages (redemption codes, users)
- Support enabling/disabling plans, configuring payment IDs, and model quotas
- Add user subscription binding modal in user management

Frontend - Wallet:
- Add subscription plans card with current subscription status display
- Show all subscriptions (active and expired) with remaining days/usage percentage
- Display purchasable plans with pricing cards following SaaS best practices
- Extract purchase modal to separate component matching payment confirm modal style
- Add skeleton loading states with active animation
- Implement billing preference selector in card header
- Handle payment gateway availability based on admin configuration

Frontend - Usage Logs:
- Display subscription deduction details in log entries
- Show step-by-step breakdown of subscription usage (pre-consumed, delta, final, remaining)
- Add subscription deduction tag for subscription-covered requests

*  feat(admin): add user subscription management and refine UI/pagination

Add admin APIs to list/create/invalidate/delete user subscriptions
Add model helpers to fetch all user subscriptions (incl. expired) and support cancel/hard-delete
Wire new admin routes for user subscription operations
Replace “Bind subscription plan” entry with a dedicated User Subscriptions SideSheet in Users table
Use CardTable with responsive layout and working client-side pagination inside the SideSheet
Improve subscription purchase modal empty-gateway state with a Banner notice

*  feat(admin): streamline subscription plan benefits editor with bulk actions

Restore the avatar/icon header for the “Model Benefits” section
Replace scattered controls with a compact toolbar-style workflow
Support multi-select add with a default quota for new items
Add row selection with bulk apply-to-selected / apply-to-all quota updates
Enable delete-selected to manage benefits faster and reduce mistakes

*  fix(subscription): finalize payments, log billing, and clean up dead code

Complete subscription orders by creating a matching top-up record and writing billing logs
Add Epay return handler to verify and finalize browser callbacks
Require Stripe/Creem webhook configuration before starting subscription payments
Show subscription purchases in topup history with clearer labels/methods
Remove unused subscription helper, legacy Creem webhook struct, and unused topup fields
Simplify subscription self API payload to active/all lists only

* 🎨 style: format all code with gofmt and lint:fix

Apply consistent code formatting across the entire codebase using
gofmt and lint:fix tools. This ensures adherence to Go community
standards and improves code readability and maintainability.

Changes include:
- Run gofmt on all .go files to standardize formatting
- Apply lint:fix to automatically resolve linting issues
- Fix code style inconsistencies and formatting violations

No functional changes were made in this commit.

*  feat(subscription): add quota reset periods and admin configuration

- Add reset period fields on subscription plans and user items
- Apply automatic quota resets during pre-consume based on plan schedule
- Expose reset-period configuration in the admin plan editor
- Display reset cadence in subscription cards and purchase modal
- Validate custom reset seconds on plan create/update

*  feat(subscription): harden subscription billing with resets, idempotency, and production-grade stability

Add plan-level quota reset periods and display/reset cadence in admin/UI
Enforce natural reset alignment with background reset task and cleanup job
Make subscription pre-consume/refund idempotent with request-scoped records and retries
Use database time for consistent resets across multi-instance deployments
Harden payment callbacks with locking and idempotent order completion
Record subscription purchases in topup history and billing logs
Optimize subscription queries and add critical composite indexes

*  feat(subscription): cache plan lookups and stabilize pre-consume

Introduce hybrid caches for subscription plans, items, and plan info with explicit
invalidation on admin updates. Streamline pre-consume transactions to reduce
redundant queries while preserving idempotency and reset logic.

* 🐛 fix(subscription): avoid pre-consume lookup noise

Use a RowsAffected check for the idempotency lookup so missing records
no longer surface as "record not found" errors while preserving behavior.

* 🔧 ci: Change workflow trigger to sub branch

Update the Docker image workflow to run on pushes to the sub branch instead of main.

* 💸 chore: Align subscription pricing display with global currency settings
Unify subscription price rendering to use the site-wide currency symbol/rate on the wallet and admin views.
Make subscription plan currency read-only in the editor and force USD on create/update to avoid drift.
Use global currency display type when creating Creem checkout payloads.

* 🔧 chore: Unify subscription plan status toggle with PATCH endpoint

Replace separate enable/disable flows with a single PATCH API that updates the enabled flag.
Update frontend hooks and table actions to call the unified endpoint and keep UI behavior consistent.
Introduce a minimal admin controller handler and route for the status update.

*  feat: Add subscription limits and UI tags consistency

Add per-plan purchase limits with backend enforcement and UI disable states.
Expose limit configuration in admin plan editor and show limits in plan tables/cards.
Refine subscription UI tags with unified badge style and streamlined “My Subscriptions” layout.

* 🎨 style: tag color to white

* 🚀 refactor: Simplify subscription quota to total amount model

Remove per-model subscription items and switch to a single total quota per plan and user subscription. Update billing, reset, and logging flows to operate on total quota, and refactor admin/user UI to configure and display total quota consistently.

* 🚀 chore: Remove duplicate subscription usage percentage display

Keep the usage percentage shown only in the total quota line to avoid redundant “已用 0%” text while preserving remaining days in the summary.

*  feat: Add subscription upgrade group with auto downgrade

*  feat: Update subscription purchase modal display

Show total quota as currency with tooltip for raw quota, hide reset cycle when never, and display upgrade group when configured to match card display rules.

*  feat: Extract quota conversion helpers to shared utils

Move quota display/conversion helpers into web/src/helpers/quota.js and update the subscription plan editor to import and use the shared utilities instead of inline functions.

*  chore: Add upgrade group guidance in subscription editor

Add explanatory helper text under the upgrade group field to clarify automatic group upgrades, rollback conditions, and the expected delay before downgrading takes effect.

* 🔧 chore: remove unused Creem settings state

Drop the unused originInputs state and redundant updates to keep the Creem
settings form state minimal and easier to maintain.

* 🚀 chore: Remove useless action

*  Add full i18n coverage for subscription-related UI across locales

*  feat: harden subscription billing and improve UI consistency

Improve subscription payment safety and data integrity by handling user/URL lookup failures, fixing Stripe subscription mode, persisting quota reset fields, and correcting subscription delta accounting and DB timestamp casting. Refine the UI with stricter custom duration validation, accurate currency rounding, conditional Epay labeling, rollback on preference update failure, and shared subscription formatting helpers plus clearer component naming.

* 🔧 fix: make epay webhook and return flow subscription-aware

Ensure Epay webhook acknowledges success only after order completion, returning fail on processing errors to allow retries. Redirect subscription payment returns to the subscription page instead of top-up for correct user flow.

* 🚦 fix: guard epay return success on order completion
Redirect subscription return flow to failure when order completion fails, preventing false success states after payment verification.

* 🔧 fix: normalize epay error handling and webhook retries

Standardize SubscriptionRequestEpay error responses via ApiErrorMsg for a consistent schema.
Return "fail" on non-success trade statuses in the epay webhook to preserve retry behavior.

* 🧾 fix: persist epay orders before purchase

Create the subscription order before initiating epay payment and expire it if the provider call fails, preventing orphaned transactions and improving reconciliation.

* 🔧 fix: harden epay callbacks and billing fallbacks

Use POST and form parsing for epay notify/return routes, persist epay orders before provider calls with expiry on failure, and ensure notify handlers retry correctly.
Restrict subscription-first fallback to insufficient-subscription errors and log refund failures after retries to avoid silent quota drift.

* 🔧 fix: harden billing flow and sidebar settings

Add missing strings import for subscription fallback checks, log failed subscription refunds after retries, and extend sidebar module settings with a subscription management toggle plus translations.

* 🛡️ fix: fail fast on epay form parse errors

Handle ParseForm errors in epay notify/return handlers by returning fail or redirecting to failure, avoiding unsafe fallback to query parameters.

*  fix: refine Japanese subscription status labels

Adjust Japanese UI wording for active-count labels to read more naturally and consistently.

*  fix: standardize epay success response schema

Return subscription epay pay success responses via ApiSuccess to include the consistent success field and align with error schema.
2026-02-03 17:40:43 +08:00
Seefs
63b642f39a Merge pull request #2745 from mehunk/feat/custom-stripe-url
feat: Support customizing the success and cancel url of Stripe.
2026-02-02 13:15:05 +08:00
Calcium-Ion
1c983a04d3 feat: disk request body cache (#2780)
* feat: 引入通用 HTTP BodyStorage/DiskCache 缓存配置与管理

- 新增 common/body_storage.go 提供 HTTP 请求体存储抽象和文件缓存能力
- 增加 common/disk_cache_config.go 支持全局磁盘缓存配置
- main.go 挂载缓存初始化流程
- 新增和补充 controller/performance.go (及 unix/windows) 用于缓存性能监控接口
- middleware/body_cleanup.go 自动清理缓存文件
- router 挂载相关接口
- 前端 settings 页面新增性能监控设置 PerformanceSetting
- 优化缓存开关状态和模块热插拔能力
- 其他相关文件同步适配缓存扩展

* fix: 修复 BodyStorage 并发安全和错误处理问题

- 修复 diskStorage.Close() 竞态条件,先获取锁再执行 CAS
- 为 memoryStorage 添加互斥锁和 closed 状态检查
- 修复 CreateBodyStorageFromReader 在磁盘存储失败时的回退逻辑
- 添加缓存命中统计调用 (IncrementDiskCacheHits/IncrementMemoryCacheHits)
- 修复 gin.go 中 Seek 错误被忽略的问题
- 在 api-router 添加 BodyStorageCleanup 中间件
- 修复前端 formatBytes 对异常值的处理

Co-authored-by: Cursor <cursoragent@cursor.com>

---------

Co-authored-by: Cursor <cursoragent@cursor.com>
2026-01-30 01:00:49 +08:00
mehunk
65fd33e3ef feat: Add trusted redirect domains. 2026-01-26 22:18:04 +09:00
Seefs
cc1da72d10 feat: openai response /v1/response/compact (#2644)
* feat: openai response /v1/response/compact

* feat: /v1/response/compact bill

* feat: /v1/response/compact

* feat: /v1/responses/compact -> codex channel

* feat: /v1/responses/compact -> codex channel

* feat: /v1/responses/compact -> codex channel

* feat: codex channel default models

* feat: compact model price

* feat: /v1/responses/comapct test
2026-01-26 20:20:16 +08:00
Seefs
7da04be52b fix: test using the correct path for rerank (#2736)
* fix: test using the correct path for rerank.

* fix: The `input` parameter for testing responses uses an array to accommodate certain channels, such as Codex, which are incompatible with single strings.
2026-01-26 19:57:15 +08:00
feitianbubu
151d7bedae feat: requestId time string use UTC 2026-01-22 08:58:35 +08:00
Seefs
af2d6ad8d2 feat: TLS_INSECURE_SKIP_VERIFY env 2026-01-15 14:43:53 +08:00
Seefs
e5cb9ac03a feat: codex channel (#2652)
* feat: codex channel

* feat: codex channel

* feat: codex oauth flow

* feat: codex refresh cred

* feat: codex usage

* fix: codex err message detail

* fix: codex setting ui

* feat: codex refresh cred task

* fix: import err

* fix: codex store must be false

* fix: chat -> responses tool call

* fix: chat -> responses tool call
2026-01-14 22:29:43 +08:00
Xyfacai
1ab0c35540 Merge pull request #2590 from xyfacai/fix/max-body-limit
fix: 设置默认max req body 为128MB
2026-01-06 21:47:12 +08:00
xiangsx
e13459f3a1 feat: add regex pattern to mask API keys in sensitive information 2026-01-05 22:44:11 +08:00
CaIon
ad61c0f89e fix(gin): update request body size check to allow zero limit 2026-01-05 18:55:24 +08:00
CaIon
b2d8ad7883 feat(init): increase maximum file download size to 64MB 2025-12-31 21:15:37 +08:00
Seefs
da1b51ac31 Merge branch 'upstream-main' into feature/pyro 2025-12-25 17:08:02 +08:00
CaIon
cc3ba39e72 feat(gin): improve request body handling and error reporting 2025-12-20 13:34:10 +08:00
CaIon
4ee595c448 feat(init): increase MaxRequestBodyMB to enhance request handling 2025-12-20 13:27:55 +08:00
Seefs
531dfb2555 docs: document pyroscope env var 2025-12-19 23:16:56 +08:00
Seefs
5ef7247eac docs: document pyroscope env var 2025-12-19 23:03:04 +08:00
Seefs
1168ddf9f9 fix: systemname 2025-12-19 22:27:35 +08:00
t0ng7u
8cb56fc319 🧹 fix: harden request-body size handling and error unwrapping
Tighten oversized request handling across relay paths and make error matching reliable.

- Align `MAX_REQUEST_BODY_MB` fallback to `32` in request body reader and decompression middleware
- Stop ignoring `GetRequestBody` errors in relay retry paths; return consistent **413** on oversized bodies (400 for other read errors)
- Add `Unwrap()` to `types.NewAPIError` so `errors.Is/As` can match wrapped underlying errors
- `go test ./...` passes
2025-12-16 18:10:00 +08:00
t0ng7u
8e3f9b1faa 🛡️ fix: prevent OOM on large/decompressed requests; skip heavy prompt meta when token count is disabled
Clamp request body size (including post-decompression) to avoid memory exhaustion caused by huge payloads/zip bombs, especially with large-context Claude requests. Add a configurable `MAX_REQUEST_BODY_MB` (default `32`) and document it.

- Enforce max request body size after gzip/br decompression via `http.MaxBytesReader`
- Add a secondary size guard in `common.GetRequestBody` and cache-safe handling
- Return **413 Request Entity Too Large** on oversized bodies in relay entry
- Avoid building large `TokenCountMeta.CombineText` when both token counting and sensitive check are disabled (use lightweight meta for pricing)
- Update READMEs (CN/EN/FR/JA) with `MAX_REQUEST_BODY_MB`
- Fix a handful of vet/formatting issues encountered during the change
- `go test ./...` passes
2025-12-16 17:00:19 +08:00
CaIon
39593052b6 feat(auth): enhance IP restriction handling with CIDR support 2025-12-15 17:24:09 +08:00
CaIon
4ea8cbd207 Revert "feat(audio): replace SysLog with logger for improved logging in GetAudioDuration"
This reverts commit e293be0138.
2025-12-14 00:04:40 +08:00
CaIon
e293be0138 feat(audio): replace SysLog with logger for improved logging in GetAudioDuration 2025-12-13 23:59:58 +08:00
CaIon
9c2483ef48 fix(audio): improve WAV duration calculation with enhanced PCM size handling 2025-12-13 23:57:32 +08:00
Seefs
fcafadc6bb feat: pyroscope integrate 2025-12-13 13:49:38 +08:00
zhiheng.wang
c992919d15 fix: correct sender format issues
- Adjust sender field format, add space to separate nickname and email address
- Ensure email header format complies with standard RFC specifications
- Fix potential email client sending exceptions (Tencent Cloud)
2025-12-12 16:19:14 +08:00
Calcium-Ion
5212fbd73d Merge pull request #2358 from seefs001/fix/regrex-repeat-compile
fix: regex repeat compile
2025-12-09 14:01:07 +08:00
CaIon
0b9f6a58bc feat: 将任务查询数量改为可配置环境变量 TASK_QUERY_LIMIT 2025-12-03 19:27:15 +08:00
Seefs
896e4ac671 fix: regex repeat compile 2025-12-03 00:41:47 +08:00
Calcium-Ion
48635360cd Merge pull request #2355 from QuantumNous/feat/optimize-token-counter
feat: refactor token estimation logic
2025-12-02 21:51:09 +08:00
CaIon
f5b409d74f feat: refactor token estimation logic
- Introduced new OpenAI text models in `common/model.go`.
- Added `IsOpenAITextModel` function to check for OpenAI text models.
- Refactored token estimation methods across various channels to use estimated prompt tokens instead of direct prompt token counts.
- Updated related functions and structures to accommodate the new token estimation approach, enhancing overall token management.
2025-12-02 21:34:39 +08:00
Papersnake
3954feb993 fix: set MaxIdleConnsPerHost to 100 2025-12-02 09:55:03 +08:00
Seefs
2520c8b25d fix: nano banana pro 4k(StreamScannerMaxBufferMB env) 2025-11-30 16:08:25 +08:00
Seefs
2f80e3fba1 Merge pull request #2261 from wzxjohn/hotfix/analytic
fix: root page does not have analytic code
2025-11-24 14:06:02 +08:00
Calcium-Ion
3f19f18dc9 Merge pull request #2278 from seefs001/fix/release-version
fix: release workflow show version
2025-11-23 23:51:32 +08:00