Compare commits
6 Commits
db87d66403
..
dev
| Author | SHA1 | Date | |
|---|---|---|---|
| 541376e9be | |||
| 8275a04903 | |||
| 062e11b1ff | |||
| fd23dfc0b9 | |||
| d4213a6ece | |||
| 8f1d1bc5f3 |
+22
@@ -18,6 +18,28 @@ ARK_VIDEO_MODEL=
|
||||
# 默认取下限 1 小时:厂商默认 48 小时对交互式应用过长,被放弃的任务会挂两天。
|
||||
# ARK_TASK_EXPIRY_SEC=3600
|
||||
|
||||
# ---- OpenAI 图片(gpt-image-2)----
|
||||
# 密钥无默认值。缺失时该适配器不装配,服务仍可启动并提供健康检查。
|
||||
OPENAI_API_KEY=your-openai-api-key
|
||||
# OPENAI_BASE_URL=https://api.openai.com/v1
|
||||
# 属于多个组织时才需要指定。
|
||||
# OPENAI_ORGANIZATION=
|
||||
# ⚠️ 与 ARK_TIMEOUT_SEC 语义不同:OpenAI 图片接口是**同步**的,一次请求里等到
|
||||
# 图片生成完才返回,故这个值**确实是生成耗时的上限**,不是短往返的上限。
|
||||
# OPENAI_TIMEOUT_SEC=180
|
||||
# 图片模型 ID,仅作兜底。同 ARK_VIDEO_MODEL:正常由 Java 侧传入任务行的快照。
|
||||
OPENAI_IMAGE_MODEL=
|
||||
# 同时执行的同步调用数上限。这同时也是「同时有多少钱在花」的上界——超出的提交
|
||||
# 排队,而不是一起打到服务商那里。
|
||||
# OPENAI_MAX_CONCURRENT=4
|
||||
# 已完成任务在内存注册表里的保留时长(秒)。**必须大于 Java 侧最长的轮询间隔**
|
||||
# (当前阶梯最长 30 秒),否则结果会在被取走前被清掉——症状是「任务成功了但查
|
||||
# 不到」,极难查。
|
||||
# OPENAI_COMPLETED_TTL_SEC=900
|
||||
# 注册表条目上限。满了以后新提交被拒(503),**不挤掉在途任务**——挤掉一个在途
|
||||
# 任务会让一次可能正在计费的调用变成无人认领的开销。
|
||||
# OPENAI_MAX_INFLIGHT=256
|
||||
|
||||
# ---- 服务身份 JWT(ADR-023 §5)----
|
||||
# 与 patbond-creation 持有的私钥配对的 RS256 公钥。注意这是一对**独立于用户
|
||||
# token 的**密钥:现有 JwtVerifier 只验签名、不校验 aud/iss,复用同一密钥对会
|
||||
|
||||
@@ -26,9 +26,52 @@ Patbond 的 AI 生成服务(Python)。负责调用 AI 服务商 API 完成
|
||||
- Java 主动轮询本服务查询进度;回调是将来的可选优化,当前不启用。
|
||||
- 请求体**不含 `userId`**(数据最小化):完成工作只需 `jobId`、生成参数与预签名上传地址。
|
||||
|
||||
## 运行
|
||||
|
||||
```bash
|
||||
cp .env.sample .env # 填入实际值;.env 已被 gitignore
|
||||
python -m app # 默认 127.0.0.1:8000
|
||||
```
|
||||
|
||||
**只监听回环**是默认:本服务只接受来自 `patbond-creation` 的内部调用,不该对外
|
||||
可达(与项目「默认拒绝、需要外部访问时经 nginx 反代」的纪律一致)。
|
||||
|
||||
探活:`GET /health`(**在 `/internal` 之外,不需要服务票**)。它会报出实际装配了
|
||||
哪些适配器——缺服务商凭证时适配器不装配,而那个症状(任务永远排队)离根因很远,
|
||||
让探活直接说出来能省很多排查时间。
|
||||
|
||||
## 接口
|
||||
|
||||
| 端点 | 说明 |
|
||||
| --- | --- |
|
||||
| `POST /internal/generation/tasks` | 提交任务,返回**服务商**任务 ID |
|
||||
| `GET /internal/generation/tasks/{provider_request_id}` | 查询任务 |
|
||||
| `GET /health` | 探活,无需服务票 |
|
||||
|
||||
**权威契约是仓库根目录的 `openapi-internal.yaml`**(手写),不是 FastAPI 自动生成
|
||||
的 `/openapi.json`(后者仅供调试,交互文档已关闭)。两者的一致性由
|
||||
`tests/api/test_contract_drift.py` 守着——比对路径、方法、字段集、枚举取值与必填项,
|
||||
**不比对字节**(那永远不会相同,只会制造一个天天红然后被 skip 绕过的门禁)。
|
||||
|
||||
### 三类失败必须分开
|
||||
|
||||
| 类别 | 形态 | 调用方该做什么 |
|
||||
| --- | --- | --- |
|
||||
| `service` | 非 2xx + `error.kind = "service"` | 我方 bug 或配置问题。除 `internal_error`/`provider_unavailable` 外不该重试 |
|
||||
| `provider` | 非 2xx + `error.kind = "provider"` + 完整分类 | **一律以 `retry` 字段为准**,不要自己解读厂商码 |
|
||||
| 任务失败 | **200** + `state = "failed"` + `failure` | 同上看 `retry` |
|
||||
|
||||
第三类刻意不走错误通道:查询一个失败的任务,**HTTP 调用是成功的**,失败的是任务。
|
||||
做成非 2xx 会让「查不到任务」与「任务失败了」难以区分。
|
||||
|
||||
判别字段是 `error.kind`,**不是 HTTP 状态码**——状态码在两类之间会撞。
|
||||
|
||||
`provider` 信封与 `failure` 的分类字段**故意完全一致**,故调用方可复用同一段处置
|
||||
逻辑,不必为「提交时被限流」与「任务失败于限流」写两套分支。
|
||||
|
||||
## 代码结构
|
||||
|
||||
分四层,依赖**单向**向内(`config` → `providers` → `domain`),由
|
||||
分层,依赖**单向**向内(`api` → `assembly`/`config` → `providers` → `domain`),由
|
||||
`tests/test_layering.py` 自动守着——靠纪律守不住的东西,靠测试才守得住。
|
||||
|
||||
```
|
||||
@@ -37,11 +80,21 @@ app/
|
||||
├── providers/
|
||||
│ ├── base.py 生成能力抽象(新增供应商 = 实现它)
|
||||
│ ├── registry.py provider_code → 适配器
|
||||
│ └── volcengine/
|
||||
│ ├── client.py HTTP 传输与鉴权(不认识「视频」)
|
||||
│ ├── errors.py 火山错误码 → 领域 Failure
|
||||
│ ├── volcengine/
|
||||
│ │ ├── client.py HTTP 传输与鉴权(不认识「视频」)
|
||||
│ │ ├── errors.py 火山错误码 → 领域 Failure
|
||||
│ │ ├── models.py 厂商请求/响应 DTO
|
||||
│ │ ├── capabilities.py 模型能力与参数限制
|
||||
│ │ ├── dimensions.py ratio+resolution → 像素对照表
|
||||
│ │ └── video.py Seedance 视频业务
|
||||
│ └── openai/
|
||||
│ ├── client.py HTTP 传输与鉴权(JSON + multipart)
|
||||
│ ├── errors.py OpenAI 错误 → 领域 Failure
|
||||
│ ├── models.py 厂商请求/响应 DTO
|
||||
│ └── video.py Seedance 视频业务
|
||||
│ ├── capabilities.py 尺寸算术约束与画质档
|
||||
│ ├── tasks.py 同步→异步桥(本服务唯一的有状态组件)
|
||||
│ └── image.py gpt-image-2 图片业务(文生图 + 图生图)
|
||||
├── api/ HTTP 端点、服务票校验、错误信封
|
||||
├── config.py 集中配置(全部旋钮从环境变量读)
|
||||
└── assembly.py 装配根(唯一同时知道配置与具体供应商的地方)
|
||||
```
|
||||
@@ -55,6 +108,10 @@ app/
|
||||
- **`client.py` 与 `video.py` 分开。** client 只管鉴权、超时、错误解包,不认识
|
||||
「视频」。以后接火山的**图片或语音**接口时新建兄弟模块复用同一个 client,
|
||||
而不是再抄一遍鉴权与错误解包——这是拆它出来的全部理由。
|
||||
- **两家的 `client.py` 刻意不共用。** 错误体形状不同(OpenAI 是
|
||||
`{"error": {message, type, code, param}}`,火山是 `{"error": {code, message}}`),
|
||||
且 OpenAI 的编辑端点走 multipart。抽一个共同基类会让两边都被对方的特例污染
|
||||
——**同一家的不同能力之间才复用传输层,不同供应商之间不复用**。
|
||||
- **适配层不读配置**,全部通过构造参数注入。否则单测得摆弄环境变量,且同一进程
|
||||
内没法跑两套不同配置的实例。
|
||||
|
||||
@@ -73,12 +130,31 @@ app/
|
||||
| 服务商 | 能力 | provider_code | 状态 |
|
||||
| --- | --- | --- | --- |
|
||||
| 火山方舟 Seedance | 视频生成 | `volcengine-seedance` | 提交与查询已实现,产物转存待存储决策 |
|
||||
| OpenAI gpt-image-2 | 图片生成(文生图 / 图生图) | `openai-gpt-image` | 同上;另见下方「同步型供应商的代价」 |
|
||||
|
||||
接口事实与错误码映射的完整取证见 patbond-doc
|
||||
`docs/development/providers/volcengine-seedance.md`。其中两条尤其要记住:
|
||||
`docs/development/providers/volcengine-seedance.md` 与
|
||||
`docs/development/providers/openai-gpt-image.md`。其中几条尤其要记住:
|
||||
|
||||
- **火山不支持幂等键**,「提交超时后不知道有没有扣费」只能由 `max_attempts` 兜住;
|
||||
- **产物 URL 只有 24 小时有效期**,因此厂商 URL 不能当存储用,必须在时限内转存。
|
||||
- **火山产物 URL 只有 24 小时有效期**,因此厂商 URL 不能当存储用,必须在时限内转存;
|
||||
- **OpenAI 的官方错误码表里全是基础设施与计费错误**,**没有任何内容审核/侵权码**
|
||||
——一次「这个提示词不能生成」只能以通用 400 到达。故 `providers/openai/errors.py`
|
||||
用 `error.param` 的有无区分「参数错」与「内容被拒」,那一条是推断并记 WARNING;
|
||||
- **`Retry-After` 目前不读**:Java 侧退避阶梯基线 30 秒、上限 10 分钟,已长于服务商
|
||||
通常给的值,忽略它只会偏慢、不会加剧限流。接它是一次跨语言契约变更。
|
||||
|
||||
### 同步型供应商的代价
|
||||
|
||||
OpenAI 的图片接口是**同步**的(没有任务 ID,也没有查询端点),由
|
||||
`providers/openai/tasks.py` 在本服务内部包成异步。两笔代价无法靠代码消除:
|
||||
|
||||
- **本服务从此持有状态。** 进程重启会丢掉所有在途图片任务(Java 侧轮询得到
|
||||
`task_not_found`,按失败处理)。也因此 **`patbond-ai` 横向扩容前必须先解决
|
||||
任务 ID 只在签发进程内有效的问题**——多副本 + 无粘性路由会让轮询落到不认识
|
||||
该 ID 的副本上。视频那侧没有这个约束,火山的任务 ID 是全局的。
|
||||
- **超时窗口无法靠重查弥补。** 超时那次调用可能已在服务商侧完成并计费,而图片
|
||||
接口没有任务 ID 可供事后查询。只能由 `max_attempts` 兜住。
|
||||
|
||||
## 配置
|
||||
|
||||
@@ -91,6 +167,12 @@ app/
|
||||
| `ARK_TIMEOUT_SEC` | `10` | **单次 HTTP 往返**超时,不是生成耗时 |
|
||||
| `ARK_VIDEO_MODEL` | _(无)_ | 仅作兜底;正常由 Java 侧传入任务行的模型快照 |
|
||||
| `ARK_TASK_EXPIRY_SEC` | `3600` | 厂商 `execution_expires_after`,区间 `[3600, 259200]`。默认取下限——厂商默认 48 小时对交互式应用过长 |
|
||||
| `OPENAI_API_KEY` | _(无)_ | OpenAI 密钥。**缺失时该适配器不装配**,服务仍可启动 |
|
||||
| `OPENAI_TIMEOUT_SEC` | `180` | ⚠️ 与 `ARK_TIMEOUT_SEC` **语义不同**:图片接口同步,这里是**生成耗时的上限** |
|
||||
| `OPENAI_IMAGE_MODEL` | _(无)_ | 仅作兜底;正常由 Java 侧传入任务行的模型快照 |
|
||||
| `OPENAI_MAX_CONCURRENT` | `4` | 同时执行的同步调用数上限,也是「同时有多少钱在花」的上界 |
|
||||
| `OPENAI_COMPLETED_TTL_SEC` | `900` | 已完成任务在内存注册表的保留时长。**须大于 Java 侧最长轮询间隔**,否则结果会在被取走前被清掉 |
|
||||
| `OPENAI_MAX_INFLIGHT` | `256` | 注册表条目上限。满了拒绝新提交(503),**不挤掉在途任务** |
|
||||
| `PATBOND_SERVICE_JWT_PUBLIC_KEY` | _(无)_ | 校验 Java 侧服务票的 RS256 公钥 |
|
||||
| `PATBOND_SERVICE_JWT_AUDIENCE` | `patbond-ai` | **必须强制校验**,否则独立密钥对失去意义 |
|
||||
|
||||
@@ -98,7 +180,9 @@ app/
|
||||
|
||||
## 本地门禁
|
||||
|
||||
与 CI 同一套命令(`.gitea/workflows/ci.yml`):
|
||||
与 CI 同一套命令(`.gitea/workflows/ci.yml`)。**逐条分开跑并读各自的退出码**——
|
||||
`ruff check . | tail -1; echo $?` 读到的是 `tail` 的退出码,会把一次真实的 lint
|
||||
失败读成通过(本仓已因此红过一次):
|
||||
|
||||
```bash
|
||||
sh scripts/check-secrets.sh --all # 凭证防泄漏全量自查
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""启动入口:`python -m app`。
|
||||
|
||||
配置全部从环境变量读(见 `app/config.py` 与 `.env.sample`)。缺服务商凭证时服务
|
||||
仍会启动,只是该适配器不装配——`/health` 会报出实际装配了哪些,便于确认。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import os
|
||||
|
||||
import uvicorn
|
||||
|
||||
from app.api.main import create_app
|
||||
|
||||
|
||||
def main() -> None:
|
||||
# 只监听回环是默认:本服务只接受来自 patbond-creation 的内部调用,
|
||||
# 不该对外可达(与项目「默认拒绝、需要外部访问时经 nginx 反代」的纪律一致)。
|
||||
host = os.environ.get("PATBOND_AI_HOST", "127.0.0.1")
|
||||
port = int(os.environ.get("PATBOND_AI_PORT", "8000"))
|
||||
uvicorn.run(create_app(), host=host, port=port)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,78 @@
|
||||
"""服务身份 JWT 校验(ADR-023 §5)。
|
||||
|
||||
本模块是**唯一**的鉴权入口。三条纪律,每条都有测试钉住:
|
||||
|
||||
1. **强制校验 `aud` 与 `typ`。** 这是独立密钥对方案唯一的意义所在——Java 侧那边
|
||||
现有的 `JwtVerifier` 只验签名、不校验 `aud`/`iss`,正是因为如此才必须用一对
|
||||
独立密钥;如果本服务也只验签名,那对独立密钥就白搭了。
|
||||
2. **用户 token 一律拒绝。** 它们由另一对密钥签发,签名校验本身就会失败;但即使
|
||||
有人误把用户密钥配到这里,缺 `aud` 与 `typ=service` 也会被拦下——两道。
|
||||
3. **缺公钥时拒绝所有请求,而不是放行。** 配置缺失必须是「全拒」而非「全过」:
|
||||
前者立刻暴露,后者是敞开的门。这与 Java 侧 `InternalAuthFilter` 未配置
|
||||
`internal-token` 时全拒 `/internal/**` 的既有选择一致。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import jwt
|
||||
from fastapi import Depends, Request
|
||||
from fastapi.security import HTTPAuthorizationCredentials, HTTPBearer
|
||||
|
||||
from app.config import ServiceAuthSettings
|
||||
|
||||
from .errors import ServiceError, ServiceErrorCode
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: auto_error=False:自己产生 401 以便统一走 service 错误信封,
|
||||
#: 而不是 FastAPI 默认的 `{"detail": ...}`——后者与本契约的两类错误形状都不符。
|
||||
_bearer = HTTPBearer(auto_error=False)
|
||||
|
||||
|
||||
def verify_service_token(
|
||||
request: Request,
|
||||
credentials: HTTPAuthorizationCredentials | None = Depends(_bearer),
|
||||
) -> None:
|
||||
"""校验服务票;不通过即抛 `ServiceError(unauthenticated)`。
|
||||
|
||||
所有失败都返回同一个笼统原因,**不告诉调用方是签名错、过期、还是 aud 不符**。
|
||||
对方是我们自己的服务,真正的原因在本服务日志里;把细节回给调用方只会在将来
|
||||
某天变成攻击者的提示。
|
||||
"""
|
||||
settings: ServiceAuthSettings = request.app.state.settings.service_auth
|
||||
|
||||
if not settings.public_key:
|
||||
# 配置缺失 → 全拒。绝不「未配置即放行」。
|
||||
log.error("PATBOND_SERVICE_JWT_PUBLIC_KEY 未配置,/internal/** 全部拒绝")
|
||||
raise ServiceError(ServiceErrorCode.UNAUTHENTICATED, "服务端未配置服务票公钥")
|
||||
|
||||
if credentials is None or not credentials.credentials:
|
||||
raise ServiceError(ServiceErrorCode.UNAUTHENTICATED, "缺少服务票")
|
||||
|
||||
try:
|
||||
claims = jwt.decode(
|
||||
credentials.credentials,
|
||||
settings.public_key,
|
||||
algorithms=["RS256"],
|
||||
# audience 与 issuer 交给库强制校验,而不是解出来自己比——
|
||||
# 自己比容易漏掉「claim 不存在」这一路。
|
||||
audience=settings.audience,
|
||||
issuer=settings.issuer,
|
||||
options={"require": ["exp", "aud", "iss", "sub"]},
|
||||
)
|
||||
except jwt.InvalidTokenError as exc:
|
||||
# 具体原因只进日志。
|
||||
log.warning("服务票校验失败:%s", exc)
|
||||
raise ServiceError(
|
||||
ServiceErrorCode.UNAUTHENTICATED, "服务票无效或已过期"
|
||||
) from exc
|
||||
|
||||
token_type = claims.get("typ")
|
||||
if token_type != settings.required_type:
|
||||
# 用户 token 走到这里的唯一可能是有人误配了用户密钥。拦住它。
|
||||
log.warning(
|
||||
"服务票 typ 不符:期望 %s,实际 %r", settings.required_type, token_type
|
||||
)
|
||||
raise ServiceError(ServiceErrorCode.UNAUTHENTICATED, "服务票类型不符")
|
||||
@@ -0,0 +1,86 @@
|
||||
"""两类错误的异常与信封(契约 `openapi-internal.yaml` 的核心设计)。
|
||||
|
||||
**两类失败必须分开**,因为调用方对它们的反应完全不同:
|
||||
|
||||
- `service`(Java↔Python):我方 bug 或配置问题。除 `internal_error` /
|
||||
`provider_unavailable` 外都不该重试——重试一个拼错的请求体不会变好。
|
||||
- `provider`(Python↔服务商):以 `retry` 字段为准,调用方不该自己解读厂商码。
|
||||
|
||||
还有**第三类不在这里**:任务本身失败(生成跑完但失败了)走 `200 + state=failed`
|
||||
+ `failure`。因为那种情况下 **HTTP 调用是成功的**,失败的是任务。做成错误响应会
|
||||
让「查不到任务」与「任务失败了」难以区分。
|
||||
|
||||
判别字段是 `error.kind`,**不是 HTTP 状态码**:状态码在两类之间会撞(502 既可能是
|
||||
服务商挂了也可能是我们打错地址),而 `kind` 不会。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from enum import Enum
|
||||
|
||||
from app.domain.generation import Failure
|
||||
|
||||
|
||||
class ServiceErrorCode(Enum):
|
||||
"""`service` 类错误码,与契约的枚举逐一对应。"""
|
||||
|
||||
#: 服务票缺失、过期、签名错、aud/typ 不符。细节只进日志。
|
||||
UNAUTHENTICATED = ("unauthenticated", 401)
|
||||
#: 请求体不合契约,或参数超出该模型的限制(本地预校验拦下)。
|
||||
INVALID_REQUEST = ("invalid_request", 400)
|
||||
#: 路径里的服务商任务 ID 不存在(含超过服务商 7 天保留期)。
|
||||
TASK_NOT_FOUND = ("task_not_found", 404)
|
||||
#: 适配器未装配(如缺服务商凭证)或 media_kind 无对应适配器。
|
||||
PROVIDER_UNAVAILABLE = ("provider_unavailable", 503)
|
||||
#: 本服务 bug。
|
||||
INTERNAL_ERROR = ("internal_error", 500)
|
||||
|
||||
def __init__(self, code: str, http_status: int):
|
||||
self.code = code
|
||||
self.http_status = http_status
|
||||
|
||||
|
||||
class ServiceError(Exception):
|
||||
"""Java↔Python 之间的问题。"""
|
||||
|
||||
def __init__(self, code: ServiceErrorCode, message: str):
|
||||
super().__init__(message)
|
||||
self.code = code
|
||||
self.message = message
|
||||
|
||||
def to_envelope(self) -> dict:
|
||||
return {
|
||||
"error": {
|
||||
"kind": "service",
|
||||
"code": self.code.code,
|
||||
"message": self.message,
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
class ProviderError(Exception):
|
||||
"""Python↔服务商之间的问题,携带完整分类。
|
||||
|
||||
字段与 `Failure` 一致**是刻意的**:调用方因此可以复用同一段处置逻辑,不必为
|
||||
「提交时被限流」和「任务失败于限流」写两套分支。
|
||||
"""
|
||||
|
||||
#: 502:本服务本身没问题,是上游调用没成功。
|
||||
#: 刻意不透传服务商的状态码——429 既可能是「等几秒」也可能是「等到明天」,
|
||||
#: 状态码表达不了这个区别,`retry` 才是权威信号。
|
||||
HTTP_STATUS = 502
|
||||
|
||||
def __init__(self, failure: Failure):
|
||||
super().__init__(failure.message or failure.provider_code)
|
||||
self.failure = failure
|
||||
|
||||
def to_envelope(self) -> dict:
|
||||
return {
|
||||
"error": {
|
||||
"kind": "provider",
|
||||
"retry": self.failure.retry.value,
|
||||
"user_facing": self.failure.user_facing.value,
|
||||
"provider_code": self.failure.provider_code,
|
||||
"message": self.failure.message,
|
||||
}
|
||||
}
|
||||
+121
@@ -0,0 +1,121 @@
|
||||
"""FastAPI 应用工厂。
|
||||
|
||||
装配顺序:读配置 → 建适配器注册表 → 挂路由与异常处理器。配置与注册表放在
|
||||
`app.state` 上,请求处理时从 `request.app` 取——**不用模块级全局**,这样测试能起
|
||||
多个配置不同的实例而互不干扰。
|
||||
|
||||
**异常处理器是本模块的重点**:契约定义了两类错误信封(`kind = service` /
|
||||
`provider`),而 FastAPI 默认的校验错误是 `{"detail": [...]}`,与两者都不符。若不
|
||||
接管,调用方就要处理三种形状——而第三种还是框架的实现细节。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import FastAPI, Request
|
||||
from fastapi.exceptions import RequestValidationError
|
||||
from fastapi.responses import JSONResponse
|
||||
|
||||
from app.assembly import build_registry
|
||||
from app.config import Settings
|
||||
|
||||
from .errors import ProviderError, ServiceError, ServiceErrorCode
|
||||
from .routes import router
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
def create_app(settings: Settings | None = None) -> FastAPI:
|
||||
"""建应用。
|
||||
|
||||
:param settings: 省略即从环境变量读。测试传入显式配置以免依赖环境。
|
||||
"""
|
||||
resolved = settings or Settings.from_env()
|
||||
logging.basicConfig(level=resolved.log_level)
|
||||
|
||||
app = FastAPI(
|
||||
title="patbond-ai 内部接口",
|
||||
version="0.1.0",
|
||||
description=(
|
||||
"patbond-creation(Java)调用本服务执行 AI 生成的内部契约。"
|
||||
"权威契约见仓库根目录的 openapi-internal.yaml——"
|
||||
"本服务自动生成的 /openapi.json 仅供调试,不是契约。"
|
||||
),
|
||||
# 内部服务不开交互文档:少一个对外表面,且契约以手写 yaml 为准。
|
||||
docs_url=None,
|
||||
redoc_url=None,
|
||||
)
|
||||
app.state.settings = resolved
|
||||
app.state.registry = build_registry(resolved)
|
||||
|
||||
app.include_router(router)
|
||||
_install_exception_handlers(app)
|
||||
_install_health(app)
|
||||
return app
|
||||
|
||||
|
||||
def _install_exception_handlers(app: FastAPI) -> None:
|
||||
@app.exception_handler(ServiceError)
|
||||
def handle_service_error(request: Request, exc: ServiceError) -> JSONResponse:
|
||||
return JSONResponse(status_code=exc.code.http_status, content=exc.to_envelope())
|
||||
|
||||
@app.exception_handler(ProviderError)
|
||||
def handle_provider_error(request: Request, exc: ProviderError) -> JSONResponse:
|
||||
log.warning(
|
||||
"服务商调用失败 code=%s retry=%s",
|
||||
exc.failure.provider_code,
|
||||
exc.failure.retry.value,
|
||||
)
|
||||
return JSONResponse(
|
||||
status_code=ProviderError.HTTP_STATUS, content=exc.to_envelope()
|
||||
)
|
||||
|
||||
@app.exception_handler(RequestValidationError)
|
||||
def handle_validation_error(
|
||||
request: Request, exc: RequestValidationError
|
||||
) -> JSONResponse:
|
||||
"""把 FastAPI 的校验错误翻成 service 信封。
|
||||
|
||||
默认形状是 `{"detail": [...]}`,与契约的两类信封都不符。不接管的话调用方
|
||||
要处理三种形状,而第三种还是框架的实现细节——换个框架就变了。
|
||||
"""
|
||||
first = exc.errors()[0] if exc.errors() else {}
|
||||
# 字段路径形如 ("body", "duration_sec"),去掉 "body" 前缀更可读。
|
||||
location = ".".join(
|
||||
str(part) for part in first.get("loc", ()) if part != "body"
|
||||
)
|
||||
detail = first.get("msg", "请求体不合契约")
|
||||
message = f"{location}: {detail}" if location else detail
|
||||
error = ServiceError(ServiceErrorCode.INVALID_REQUEST, message)
|
||||
return JSONResponse(
|
||||
status_code=error.code.http_status, content=error.to_envelope()
|
||||
)
|
||||
|
||||
@app.exception_handler(Exception)
|
||||
def handle_unexpected(request: Request, exc: Exception) -> JSONResponse:
|
||||
"""兜底:本服务的 bug。
|
||||
|
||||
完整堆栈只进日志,绝不回给调用方——内部细节外泄没有任何好处。
|
||||
"""
|
||||
log.exception("未处理的异常")
|
||||
error = ServiceError(ServiceErrorCode.INTERNAL_ERROR, "服务内部错误")
|
||||
return JSONResponse(
|
||||
status_code=error.code.http_status, content=error.to_envelope()
|
||||
)
|
||||
|
||||
|
||||
def _install_health(app: FastAPI) -> None:
|
||||
@app.get("/health", include_in_schema=False)
|
||||
def health() -> dict:
|
||||
"""探活。**在 /internal 之外,故不需要服务票**——与 Java 侧各服务
|
||||
`/health` 不挂鉴权的既有约定一致。
|
||||
|
||||
同时报出已装配的适配器:缺凭证时适配器不装配,而那个症状(任务永远排队)
|
||||
离根因很远,让探活直接说出来能省很多排查时间。
|
||||
"""
|
||||
registry = app.state.registry
|
||||
return {
|
||||
"status": "ok",
|
||||
"providers": sorted(registry.codes),
|
||||
}
|
||||
@@ -0,0 +1,123 @@
|
||||
"""内部接口路由。
|
||||
|
||||
**本层只做编排,不含任何厂商知识**:解析请求 → 按 `media_kind` 找适配器 → 调用 →
|
||||
把领域结果序列化。厂商差异全在 `app/providers/<厂商>/` 内。
|
||||
|
||||
一处刻意的选择:路径参数是**服务商的任务 ID**,不是我方 `job_id`。本服务不持有
|
||||
状态,唯一能用来问服务商的就是这个 ID;若用 `job_id` 就得存一份映射,「无状态」
|
||||
不再成立,且会多出一个「Java 拿到 ID 但查不到任何东西」的中间态。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, Depends, Request
|
||||
|
||||
from app.domain.generation import MediaKind
|
||||
from app.providers.base import GenerationProvider
|
||||
from app.providers.openai.client import OpenAiError
|
||||
from app.providers.registry import ProviderRegistry
|
||||
from app.providers.volcengine.client import ArkError
|
||||
|
||||
from .auth import verify_service_token
|
||||
from .errors import ProviderError, ServiceError, ServiceErrorCode
|
||||
from .schemas import SubmitRequest, SubmitResponse, TaskResponse
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
router = APIRouter(
|
||||
prefix="/internal/generation",
|
||||
dependencies=[Depends(verify_service_token)],
|
||||
)
|
||||
|
||||
|
||||
def _registry(request: Request) -> ProviderRegistry:
|
||||
return request.app.state.registry
|
||||
|
||||
|
||||
def _provider_for(
|
||||
registry: ProviderRegistry, media_kind: MediaKind
|
||||
) -> GenerationProvider:
|
||||
"""按 `media_kind` 取适配器。
|
||||
|
||||
查不到即 `provider_unavailable`(503)而不是 500:这通常是凭证未配置导致适配器
|
||||
没装配,属环境问题而非代码缺陷,且对调用方而言是**可重试**的——配好就能用。
|
||||
"""
|
||||
for provider in registry.all():
|
||||
if provider.media_kind is media_kind:
|
||||
return provider
|
||||
raise ServiceError(
|
||||
ServiceErrorCode.PROVIDER_UNAVAILABLE,
|
||||
f"没有可用的 {media_kind.value} 生成适配器(检查服务商凭证是否已配置)",
|
||||
)
|
||||
|
||||
|
||||
@router.post("/tasks", response_model=SubmitResponse)
|
||||
def submit_task(payload: SubmitRequest, request: Request) -> SubmitResponse:
|
||||
"""提交生成任务,返回服务商任务 ID。
|
||||
|
||||
调用方**必须在上游产生任何计费动作之前**把该 ID 落库——火山不支持幂等键,
|
||||
这是提交超时后唯一能判断「是否已提交」的依据。
|
||||
"""
|
||||
provider = _provider_for(_registry(request), payload.media_kind)
|
||||
try:
|
||||
provider_request_id = provider.submit(payload.to_domain())
|
||||
except (ArkError, OpenAiError) as exc:
|
||||
# 服务商调用失败:原样带上分类,调用方据 retry 决定重试。
|
||||
raise ProviderError(exc.failure) from exc
|
||||
except ValueError as exc:
|
||||
# 本地预校验拦下的参数问题(超出该模型的时长/尺寸/ratio 限制、内容为空、
|
||||
# 参考图不可下载)。属 invalid_request 而非 provider 错误——请求根本没发出去。
|
||||
raise ServiceError(ServiceErrorCode.INVALID_REQUEST, str(exc)) from exc
|
||||
except RuntimeError as exc:
|
||||
# 同步型适配器的在途任务已达上限。这不是请求的问题而是容量问题,且稍后
|
||||
# 重试会好,故归 provider_unavailable(503)而非 invalid_request——后者会
|
||||
# 让调用方判为不可重试并直接失败掉一个本可成功的任务。
|
||||
raise ServiceError(ServiceErrorCode.PROVIDER_UNAVAILABLE, str(exc)) from exc
|
||||
|
||||
log.info(
|
||||
"已提交生成任务 job_id=%s provider_request_id=%s",
|
||||
payload.job_id,
|
||||
provider_request_id,
|
||||
)
|
||||
return SubmitResponse(provider_request_id=provider_request_id)
|
||||
|
||||
|
||||
@router.get("/tasks/{provider_request_id}", response_model=TaskResponse)
|
||||
def get_task(
|
||||
provider_request_id: str,
|
||||
request: Request,
|
||||
media_kind: MediaKind = MediaKind.VIDEO,
|
||||
) -> TaskResponse:
|
||||
"""查询任务。
|
||||
|
||||
**任务失败也返回 200**:HTTP 调用成功了,失败的是任务,`state=failed` +
|
||||
`failure` 表达它。做成非 2xx 会让「查不到任务」与「任务失败了」难以区分。
|
||||
|
||||
:param media_kind: 决定问哪个适配器。默认 video——当前只有视频适配器;
|
||||
接入图片后调用方应显式传入。刻意不做「遍历所有适配器试一遍」:那会把
|
||||
一个任务 ID 拿去问不相干的服务商。
|
||||
"""
|
||||
provider = _provider_for(_registry(request), media_kind)
|
||||
try:
|
||||
result = provider.fetch(provider_request_id)
|
||||
except KeyError as exc:
|
||||
# 同步型适配器的本地注册表里没有这个 ID:从未存在过,或进程重启后丢失
|
||||
# (两者无法区分)。两种情形下调用方都该**停止轮询**而非退避重试。
|
||||
raise ServiceError(
|
||||
ServiceErrorCode.TASK_NOT_FOUND,
|
||||
f"本服务无此任务(可能因进程重启而丢失):{provider_request_id}",
|
||||
) from exc
|
||||
except ArkError as exc:
|
||||
# 服务商说任务不存在(含超过 7 天保留期)→ 这是 service 类的 task_not_found,
|
||||
# 而不是 provider 错误:调用方要做的是停止轮询,不是退避重试。
|
||||
if exc.http_status == 404:
|
||||
raise ServiceError(
|
||||
ServiceErrorCode.TASK_NOT_FOUND,
|
||||
f"服务商无此任务(或已超出其 7 天保留期):{provider_request_id}",
|
||||
) from exc
|
||||
raise ProviderError(exc.failure) from exc
|
||||
except OpenAiError as exc:
|
||||
raise ProviderError(exc.failure) from exc
|
||||
return TaskResponse.of(result)
|
||||
@@ -0,0 +1,142 @@
|
||||
"""请求/响应的 pydantic 模型。
|
||||
|
||||
字段名为 **snake_case**,与契约一致。这使 JSON、pydantic 模型、领域 dataclass
|
||||
三者字段逐一对应,**无需任何转换层**——若用 camelCase,两端都要加转换。
|
||||
|
||||
这层与 `app/domain/generation.py` 的分工:领域模型是纯数据、无框架依赖(那是
|
||||
「换供应商不动上层」的支点);本层负责 HTTP 边界的校验与序列化。两者字段重合,
|
||||
但让领域层依赖 pydantic 会把框架拖进本该最干净的一层。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from pydantic import BaseModel, ConfigDict, Field, model_validator
|
||||
|
||||
from app.domain.generation import (
|
||||
GenerationRequest,
|
||||
GenerationResult,
|
||||
MediaKind,
|
||||
)
|
||||
|
||||
|
||||
class SubmitRequest(BaseModel):
|
||||
"""提交生成任务。
|
||||
|
||||
**刻意不含 `user_id`**(数据最小化,ADR-023 §5):本服务干活只需要任务参数。
|
||||
需要给服务商做滥用检测时传的是 `safety_identifier`——一个哈希,不是明文 ID。
|
||||
"""
|
||||
|
||||
# 拒绝未声明的字段:调用方拼错字段名时应当立刻知道,而不是让那个参数被静默
|
||||
# 忽略——后者会表现为「我传了 duration 却没生效」这类极难查的问题。
|
||||
model_config = ConfigDict(extra="forbid")
|
||||
|
||||
job_id: str
|
||||
media_kind: MediaKind
|
||||
model: str = Field(min_length=1)
|
||||
prompt: str | None = Field(default=None, max_length=10000)
|
||||
input_url: str | None = None
|
||||
ratio: str | None = None
|
||||
resolution: str | None = None
|
||||
#: 图片路径用像素尺寸;视频路径用 ratio+resolution。两家服务商的尺寸表达方式
|
||||
#: 不同,契约同时保留两种而不强行统一——任何统一都会在某一侧失真。
|
||||
width_px: int | None = None
|
||||
height_px: int | None = None
|
||||
duration_sec: int | None = None
|
||||
generate_audio: bool = True
|
||||
watermark: bool = False
|
||||
safety_identifier: str | None = Field(default=None, max_length=64)
|
||||
callback_url: str | None = None
|
||||
upload_target: str | None = None
|
||||
|
||||
@model_validator(mode="after")
|
||||
def require_prompt_or_input(self) -> SubmitRequest:
|
||||
"""`prompt` 与 `input_url` 至少要有一个。
|
||||
|
||||
空内容必然被服务商拒,但那要花一次往返、还可能计入 RPM。本地先拦。
|
||||
"""
|
||||
if not self.prompt and not self.input_url:
|
||||
raise ValueError("prompt 与 input_url 至少要有一个")
|
||||
return self
|
||||
|
||||
def to_domain(self) -> GenerationRequest:
|
||||
return GenerationRequest(
|
||||
job_id=self.job_id,
|
||||
media_kind=self.media_kind,
|
||||
model=self.model,
|
||||
prompt=self.prompt,
|
||||
input_url=self.input_url,
|
||||
ratio=self.ratio,
|
||||
resolution=self.resolution,
|
||||
width_px=self.width_px,
|
||||
height_px=self.height_px,
|
||||
duration_sec=self.duration_sec,
|
||||
generate_audio=self.generate_audio,
|
||||
watermark=self.watermark,
|
||||
safety_identifier=self.safety_identifier,
|
||||
callback_url=self.callback_url,
|
||||
upload_target=self.upload_target,
|
||||
)
|
||||
|
||||
|
||||
class SubmitResponse(BaseModel):
|
||||
provider_request_id: str
|
||||
|
||||
|
||||
class FailureModel(BaseModel):
|
||||
"""失败分类。三个字段各有明确消费方,见契约说明。"""
|
||||
|
||||
retry: str
|
||||
user_facing: str
|
||||
provider_code: str
|
||||
message: str = ""
|
||||
|
||||
|
||||
class TaskResponse(BaseModel):
|
||||
"""任务当前状态。
|
||||
|
||||
注意 `state` 的取值集**比服务商的更小**:服务商的 `queued` 映射为 `pending`、
|
||||
`expired` 归入 `failed`(它是超时,属失败的一种,重试策略由 `failure` 表达)。
|
||||
"""
|
||||
|
||||
provider_request_id: str
|
||||
state: str
|
||||
progress: int
|
||||
asset_url: str | None = None
|
||||
asset_url_expires_in_sec: int | None = None
|
||||
cover_url: str | None = None
|
||||
ratio: str | None = None
|
||||
resolution: str | None = None
|
||||
width_px: int | None = None
|
||||
height_px: int | None = None
|
||||
duration_ms: int | None = None
|
||||
output_format: str | None = None
|
||||
billing_tokens: int | None = None
|
||||
failure: FailureModel | None = None
|
||||
|
||||
@classmethod
|
||||
def of(cls, result: GenerationResult) -> TaskResponse:
|
||||
return cls(
|
||||
provider_request_id=result.provider_request_id,
|
||||
state=result.state.value,
|
||||
progress=result.progress,
|
||||
asset_url=result.asset_url,
|
||||
asset_url_expires_in_sec=result.asset_url_expires_in_sec,
|
||||
cover_url=result.cover_url,
|
||||
ratio=result.ratio,
|
||||
resolution=result.resolution,
|
||||
width_px=result.width_px,
|
||||
height_px=result.height_px,
|
||||
duration_ms=result.duration_ms,
|
||||
output_format=result.output_format,
|
||||
billing_tokens=result.billing_tokens,
|
||||
failure=(
|
||||
FailureModel(
|
||||
retry=result.failure.retry.value,
|
||||
user_facing=result.failure.user_facing.value,
|
||||
provider_code=result.failure.provider_code,
|
||||
message=result.failure.message,
|
||||
)
|
||||
if result.failure
|
||||
else None
|
||||
),
|
||||
)
|
||||
@@ -14,6 +14,9 @@ import logging
|
||||
|
||||
from app.config import Settings
|
||||
from app.providers.base import GenerationProvider
|
||||
from app.providers.openai.client import OpenAiClient
|
||||
from app.providers.openai.image import OpenAiImageProvider
|
||||
from app.providers.openai.tasks import SyncTaskRegistry
|
||||
from app.providers.registry import ProviderRegistry
|
||||
from app.providers.volcengine.client import ArkClient
|
||||
from app.providers.volcengine.video import SeedanceVideoProvider
|
||||
@@ -48,6 +51,29 @@ def build_registry(settings: Settings) -> ProviderRegistry:
|
||||
# 与它相隔很远。
|
||||
log.warning("ARK_API_KEY 未配置,火山方舟视频适配器未装配")
|
||||
|
||||
openai = settings.openai
|
||||
if openai.api_key:
|
||||
providers.append(
|
||||
OpenAiImageProvider(
|
||||
OpenAiClient(
|
||||
api_key=openai.api_key,
|
||||
base_url=openai.base_url,
|
||||
timeout_sec=openai.timeout_sec,
|
||||
organization=openai.organization or None,
|
||||
),
|
||||
# 同步→异步的任务注册表。每个适配器实例一份:它承载的是「本进程
|
||||
# 内哪些同步调用正在跑」,共享一份会让两个适配器互相挤占容量。
|
||||
SyncTaskRegistry(
|
||||
max_workers=openai.max_concurrent,
|
||||
completed_ttl_sec=openai.completed_ttl_sec,
|
||||
max_entries=openai.max_inflight,
|
||||
),
|
||||
default_model=openai.image_model,
|
||||
)
|
||||
)
|
||||
else:
|
||||
log.warning("OPENAI_API_KEY 未配置,图片生成适配器未装配")
|
||||
|
||||
registry = ProviderRegistry(providers)
|
||||
log.info("已装配 %d 个生成适配器:%s", len(registry), sorted(registry.codes))
|
||||
return registry
|
||||
|
||||
@@ -14,6 +14,7 @@ from __future__ import annotations
|
||||
import os
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from app.providers.openai.client import DEFAULT_BASE_URL as OPENAI_DEFAULT_BASE_URL
|
||||
from app.providers.volcengine.models import DEFAULT_BASE_URL
|
||||
|
||||
|
||||
@@ -83,6 +84,59 @@ class VolcengineSettings:
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class OpenAiSettings:
|
||||
"""OpenAI 图片接入配置。
|
||||
|
||||
:param api_key: `OPENAI_API_KEY`。**无默认值**,缺失时该适配器不装配。
|
||||
:param timeout_sec: **这里确实是生成耗时的上限**,与火山那边不同——OpenAI 的
|
||||
图片接口是同步的,一次请求里等到图片生成完才返回。故默认值比火山的
|
||||
「短往返超时」大一个量级。
|
||||
:param image_model: 图片模型 ID,仅作兜底;正常由 Java 侧传入任务行的快照。
|
||||
:param max_concurrent: 同时执行的同步调用数上限。**同时也是对服务商并发和
|
||||
「同时有多少钱在花」的自我限制**——超出的提交排队而非一起打过去。
|
||||
:param completed_ttl_sec: 已完成任务在内存注册表里的保留时长。须大于 Java 侧
|
||||
最长的轮询间隔(当前阶梯最长 30 秒),否则结果会在被取走前被清理。
|
||||
"""
|
||||
|
||||
api_key: str = ""
|
||||
base_url: str = OPENAI_DEFAULT_BASE_URL
|
||||
timeout_sec: float = 180.0
|
||||
image_model: str = ""
|
||||
organization: str = ""
|
||||
max_concurrent: int = 4
|
||||
completed_ttl_sec: float = 900.0
|
||||
max_inflight: int = 256
|
||||
|
||||
@classmethod
|
||||
def from_env(cls) -> OpenAiSettings:
|
||||
return cls(
|
||||
api_key=_env("OPENAI_API_KEY"),
|
||||
base_url=_env("OPENAI_BASE_URL", cls.base_url),
|
||||
timeout_sec=_env_float("OPENAI_TIMEOUT_SEC", cls.timeout_sec),
|
||||
image_model=_env("OPENAI_IMAGE_MODEL"),
|
||||
organization=_env("OPENAI_ORGANIZATION"),
|
||||
max_concurrent=_env_int("OPENAI_MAX_CONCURRENT", cls.max_concurrent),
|
||||
completed_ttl_sec=_env_float(
|
||||
"OPENAI_COMPLETED_TTL_SEC", cls.completed_ttl_sec
|
||||
),
|
||||
max_inflight=_env_int("OPENAI_MAX_INFLIGHT", cls.max_inflight),
|
||||
)
|
||||
|
||||
def __post_init__(self) -> None:
|
||||
if self.max_concurrent < 1:
|
||||
raise ValueError(
|
||||
f"OPENAI_MAX_CONCURRENT 必须 >= 1,当前 {self.max_concurrent}"
|
||||
)
|
||||
# 小于最长轮询间隔会让结果在被取走前就被清掉——那表现为「任务成功了但查
|
||||
# 不到」,极难查。
|
||||
if self.completed_ttl_sec < 60:
|
||||
raise ValueError(
|
||||
f"OPENAI_COMPLETED_TTL_SEC 不应小于 60 秒(须大于 Java 侧最长轮询"
|
||||
f"间隔,否则结果会在被取走前被清理),当前 {self.completed_ttl_sec}"
|
||||
)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ServiceAuthSettings:
|
||||
"""校验 Java 侧服务身份 JWT 所需的配置(ADR-023 §5)。
|
||||
@@ -113,6 +167,7 @@ class Settings:
|
||||
"""本服务的全部配置。"""
|
||||
|
||||
volcengine: VolcengineSettings = field(default_factory=VolcengineSettings)
|
||||
openai: OpenAiSettings = field(default_factory=OpenAiSettings)
|
||||
service_auth: ServiceAuthSettings = field(default_factory=ServiceAuthSettings)
|
||||
log_level: str = "INFO"
|
||||
|
||||
@@ -120,6 +175,7 @@ class Settings:
|
||||
def from_env(cls) -> Settings:
|
||||
return cls(
|
||||
volcengine=VolcengineSettings.from_env(),
|
||||
openai=OpenAiSettings.from_env(),
|
||||
service_auth=ServiceAuthSettings.from_env(),
|
||||
log_level=_env("PATBOND_AI_LOG_LEVEL", cls.log_level).upper(),
|
||||
)
|
||||
|
||||
@@ -109,6 +109,12 @@ class GenerationRequest:
|
||||
input_url: str | None = None
|
||||
ratio: str | None = None
|
||||
resolution: str | None = None
|
||||
#: 像素尺寸。**图片路径用它,视频路径用 ratio+resolution**——两家服务商的
|
||||
#: 尺寸表达方式不同:火山用「分辨率档 + 宽高比」,OpenAI 用 WIDTHxHEIGHT。
|
||||
#: 领域层同时保留两种表达而不强行统一,是因为任何统一都会在某一侧失真
|
||||
#: (把 720p+16:9 折成像素会丢掉"档位"语义,把像素折成比例会丢掉精确值)。
|
||||
width_px: int | None = None
|
||||
height_px: int | None = None
|
||||
duration_sec: int | None = None
|
||||
generate_audio: bool = True
|
||||
watermark: bool = False
|
||||
|
||||
@@ -0,0 +1,151 @@
|
||||
"""gpt-image-2 的尺寸与参数约束。
|
||||
|
||||
与火山那边不同,OpenAI 的图片接口不用「分辨率 + 宽高比」两个字段,而是一个
|
||||
`WIDTHxHEIGHT` 字符串,并对它施加几条**算术约束**。逐条取自官方文档:
|
||||
|
||||
- 宽高**都必须能被 16 整除**
|
||||
- 宽高比必须在 `1:3` 到 `3:1` 之间
|
||||
- 最大支持 `3840x2160`
|
||||
- 超过 `2560x1440` 属**实验性**
|
||||
- 另有「模型当前的像素与边长限制」——文档未给具体数值,故无法在本地校验
|
||||
|
||||
在本地先校验这些,是因为违反约束的请求会被 OpenAI 以 400 拒绝,而那要花一次
|
||||
往返、计入配额,并表现为一个失败的任务。错误信息还能直接指出违反了哪一条。
|
||||
|
||||
**标准尺寸**(`1024x1024` / `1536x1024` / `1024x1536`)与 `auto` 一律放行,
|
||||
不做算术校验:它们是文档明示支持的取值,其中 `auto` 由模型自行决定。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import re
|
||||
from dataclasses import dataclass
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: 文档明示的标准尺寸,直接放行。
|
||||
STANDARD_SIZES = frozenset({"1024x1024", "1536x1024", "1024x1536"})
|
||||
|
||||
#: 交给模型自选尺寸。
|
||||
AUTO_SIZE = "auto"
|
||||
|
||||
#: `WIDTHxHEIGHT`。
|
||||
_SIZE_PATTERN = re.compile(r"^(\d+)x(\d+)$")
|
||||
|
||||
#: 宽高必须同时是它的倍数。
|
||||
EDGE_MULTIPLE = 16
|
||||
|
||||
#: 宽高比的允许区间(宽/高),即 1:3 ~ 3:1。
|
||||
MIN_ASPECT = 1 / 3
|
||||
MAX_ASPECT = 3.0
|
||||
|
||||
#: 最大支持尺寸,文档原文是「the maximum supported resolution is 3840x2160」。
|
||||
#:
|
||||
#: ⚠️ **这里有一处文档未明确、由我们推断的语义**:那句话没说清限制是「逐边」
|
||||
#: (宽≤3840 且 高≤2160)还是「按总像素」。按逐边理解会**禁掉所有高于 2160 的
|
||||
#: 竖图**,而 `1024x1536` 是文档明示的标准竖图尺寸、竖版显然受支持——故逐边理解
|
||||
#: 必然过严。
|
||||
#:
|
||||
#: 采用的读法:**最长边 ≤ 3840,且总像素 ≤ 3840×2160**。它允许 `2160x3840`
|
||||
#: 这类竖图(与横图同像素量),又拦住明显越界的尺寸。若日后拿到更明确的文档或
|
||||
#: 实测结果,回来校正本处并更新取证页。
|
||||
MAX_EDGE = 3840
|
||||
MAX_PIXELS = 3840 * 2160
|
||||
|
||||
#: 超过此面积属实验性——放行但记一条日志,便于把偶发的异常表现与它关联起来。
|
||||
#: 同样按面积而非逐边理解,理由同上。
|
||||
EXPERIMENTAL_ABOVE_PIXELS = 2560 * 1440
|
||||
|
||||
#: 可选的画质档。xhigh 与 max 仅 2.5 系列支持,gpt-image-2 不含。
|
||||
QUALITIES_GPT_IMAGE_2 = frozenset({"low", "medium", "high", "auto"})
|
||||
|
||||
#: 输出格式。注意 transparent 背景要求 png 或 webp。
|
||||
OUTPUT_FORMATS = frozenset({"png", "jpeg", "webp"})
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class Size:
|
||||
width: int
|
||||
height: int
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f"{self.width}x{self.height}"
|
||||
|
||||
|
||||
def parse_size(value: str) -> Size | None:
|
||||
"""把 `WIDTHxHEIGHT` 解析成尺寸;标准尺寸与 `auto` 之外的非法串返回 None。"""
|
||||
match = _SIZE_PATTERN.match(value.strip())
|
||||
if not match:
|
||||
return None
|
||||
return Size(int(match.group(1)), int(match.group(2)))
|
||||
|
||||
|
||||
def validate_size(value: str | None) -> None:
|
||||
"""校验尺寸,不合法则抛 `ValueError`(消息指明违反了哪一条约束)。
|
||||
|
||||
`None`、`auto`、标准尺寸一律放行。
|
||||
"""
|
||||
if not value or value == AUTO_SIZE or value in STANDARD_SIZES:
|
||||
return
|
||||
|
||||
size = parse_size(value)
|
||||
if size is None:
|
||||
raise ValueError(
|
||||
f"size 形态非法:{value!r},应为 'WIDTHxHEIGHT'、'auto' "
|
||||
f"或标准尺寸 {sorted(STANDARD_SIZES)}"
|
||||
)
|
||||
|
||||
if size.width % EDGE_MULTIPLE or size.height % EDGE_MULTIPLE:
|
||||
raise ValueError(f"size {size} 的宽高必须都能被 {EDGE_MULTIPLE} 整除")
|
||||
|
||||
# 宽高比先判:它是文档最明确的一条,且比尺寸上限更常被违反(比例写错比
|
||||
# 尺寸写大更容易发生),先报它的错误信息更有用。
|
||||
aspect = size.width / size.height
|
||||
if not MIN_ASPECT <= aspect <= MAX_ASPECT:
|
||||
raise ValueError(f"size {size} 的宽高比 {aspect:.2f} 超出允许区间 1:3 ~ 3:1")
|
||||
|
||||
if max(size.width, size.height) > MAX_EDGE:
|
||||
raise ValueError(f"size {size} 的最长边超出上限 {MAX_EDGE}")
|
||||
|
||||
pixels = size.width * size.height
|
||||
if pixels > MAX_PIXELS:
|
||||
raise ValueError(
|
||||
f"size {size} 的总像素 {pixels} 超出上限 {MAX_PIXELS}"
|
||||
f"(即 3840x2160 的像素量)"
|
||||
)
|
||||
|
||||
if pixels > EXPERIMENTAL_ABOVE_PIXELS:
|
||||
# 放行但记一条:文档称实验性,出问题时这条日志能把现象与它关联起来。
|
||||
log.info("size %s 超过 2560x1440,属服务商标注的实验性范围", size)
|
||||
|
||||
|
||||
def validate_quality(value: str | None) -> None:
|
||||
"""校验画质档。
|
||||
|
||||
只覆盖 gpt-image-2:`xhigh`/`max` 是 2.5 系列才支持的档,传给 gpt-image-2
|
||||
会被拒。不按模型名分表是因为目前只接入一个图片模型——接入第二个时再分。
|
||||
"""
|
||||
if value and value not in QUALITIES_GPT_IMAGE_2:
|
||||
raise ValueError(
|
||||
f"quality {value!r} 不被 gpt-image-2 支持,"
|
||||
f"可选:{sorted(QUALITIES_GPT_IMAGE_2)}"
|
||||
f"(xhigh / max 仅 gpt-image-2.5 系列支持)"
|
||||
)
|
||||
|
||||
|
||||
def validate_output_format(value: str | None, *, transparent: bool = False) -> None:
|
||||
"""校验输出格式。
|
||||
|
||||
:param transparent: 背景为透明时**只能** png 或 webp——jpeg 没有 alpha 通道,
|
||||
指定它会得到一张背景被填成不透明的图,而那不会报错、只会静默不符合预期。
|
||||
"""
|
||||
if value and value not in OUTPUT_FORMATS:
|
||||
raise ValueError(
|
||||
f"output_format {value!r} 非法,可选:{sorted(OUTPUT_FORMATS)}"
|
||||
)
|
||||
if transparent and value == "jpeg":
|
||||
raise ValueError(
|
||||
"背景为 transparent 时 output_format 不能是 jpeg(无 alpha 通道),"
|
||||
"请用 png 或 webp"
|
||||
)
|
||||
@@ -0,0 +1,166 @@
|
||||
"""OpenAI 图片接口的 HTTP 传输层。
|
||||
|
||||
与火山那边的 `ArkClient` 职责相同(鉴权、超时、错误解包),但**刻意不共用**:
|
||||
两家的错误体形状不同(OpenAI 是 `{"error": {message, type, code, param}}`,火山是
|
||||
`{"error": {code, message}}`),且 OpenAI 的图片编辑走 multipart 而非 JSON。
|
||||
强行抽一个共同基类会让两边都被对方的特例污染——供应商适配层之间**不该**互相
|
||||
复用,各自复用自家的传输层才是分层的本意。
|
||||
|
||||
本模块不认识「图片生成」这个业务:以后接 OpenAI 的对话或语音接口时复用它。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from typing import Any, Self
|
||||
|
||||
import httpx
|
||||
|
||||
from app.domain.generation import Failure
|
||||
|
||||
from .errors import classify
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: 官方 API base URL。
|
||||
DEFAULT_BASE_URL = "https://api.openai.com/v1"
|
||||
|
||||
|
||||
class OpenAiError(RuntimeError):
|
||||
"""OpenAI 返回的错误,携带领域分类。"""
|
||||
|
||||
def __init__(self, failure: Failure, http_status: int | None = None):
|
||||
super().__init__(failure.message or failure.provider_code or "OpenAI 调用失败")
|
||||
self.failure = failure
|
||||
self.http_status = http_status
|
||||
|
||||
|
||||
class OpenAiClient:
|
||||
"""OpenAI API 的同步 HTTP 客户端。
|
||||
|
||||
**超时语义与火山那边不同,这里必须说清**:OpenAI 的图片接口是**同步**的——
|
||||
一次请求里等到图片生成完再返回。因此这里的 `timeout_sec` **确实是生成耗时的
|
||||
上限**,不是「短往返」的上限,默认值也因此比火山那边大一个量级。
|
||||
|
||||
这也是 `image.py` 必须在服务内部把它转成异步的原因(ADR-023 §4:同步型供应商
|
||||
由其适配层在内部消化差异,绝不让它渗透到对外接口)。
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
api_key: str,
|
||||
base_url: str = DEFAULT_BASE_URL,
|
||||
timeout_sec: float = 180.0,
|
||||
organization: str | None = None,
|
||||
client: httpx.Client | None = None,
|
||||
):
|
||||
if not api_key:
|
||||
raise ValueError(
|
||||
"OPENAI_API_KEY 未配置:请通过环境变量注入,切勿写入代码或样例文件"
|
||||
)
|
||||
self._api_key = api_key
|
||||
self._base_url = base_url.rstrip("/")
|
||||
self._organization = organization
|
||||
self._owns_client = client is None
|
||||
self._client = client or httpx.Client(timeout=timeout_sec)
|
||||
|
||||
def post_json(self, path: str, body: dict[str, Any]) -> dict[str, Any]:
|
||||
return self._send("POST", path, json=body)
|
||||
|
||||
def post_multipart(
|
||||
self,
|
||||
path: str,
|
||||
data: dict[str, Any],
|
||||
files: list[tuple[str, tuple[str, bytes, str]]],
|
||||
) -> dict[str, Any]:
|
||||
"""multipart 提交,图像编辑接口需要它(`image` 是文件而非 URL)。
|
||||
|
||||
:param files: `(field_name, (filename, content, content_type))`。用 list
|
||||
而非 dict 是因为**同名字段可以重复**——编辑接口支持传多张图,它们
|
||||
共用字段名 `image[]`,dict 表达不了。
|
||||
"""
|
||||
return self._send("POST", path, data=data, files=files)
|
||||
|
||||
# ---------------------------------------------------------------- 内部
|
||||
|
||||
def _send(self, method: str, path: str, **kwargs: Any) -> dict[str, Any]:
|
||||
url = f"{self._base_url}{path}"
|
||||
try:
|
||||
response = self._client.request(
|
||||
method, url, headers=self._headers(), **kwargs
|
||||
)
|
||||
except httpx.HTTPError as exc:
|
||||
# 网络层失败按可重试处理。注意**同步接口的超时最危险**:请求可能已经
|
||||
# 在服务商侧完成并计费,而我们拿不到结果。OpenAI 的图片接口没有任务
|
||||
# ID 可供事后查询,故这个窗口无法通过"再查一次"弥补——只能由
|
||||
# max_attempts 兜住。详见 image.py 的说明。
|
||||
raise OpenAiError(
|
||||
classify(None, message=f"请求 OpenAI 失败:{exc}")
|
||||
) from exc
|
||||
return self._unwrap(response)
|
||||
|
||||
def _headers(self) -> dict[str, str]:
|
||||
headers = {"Authorization": f"Bearer {self._api_key}"}
|
||||
if self._organization:
|
||||
headers["OpenAI-Organization"] = self._organization
|
||||
# 刻意不设 Content-Type:JSON 与 multipart 的取值不同,交给 httpx 按
|
||||
# 实际载荷推断(multipart 还需要 boundary,手写必错)。
|
||||
return headers
|
||||
|
||||
@staticmethod
|
||||
def _unwrap(response: httpx.Response) -> dict[str, Any]:
|
||||
"""把响应转成 dict;非 2xx 转成带分类的 `OpenAiError`。
|
||||
|
||||
错误体形状取自官方 Error codes 页:`{"error": {message, type, code, param}}`。
|
||||
四个字段**都要取**——`param` 是区分「参数错」与「内容被拒」的唯一结构化
|
||||
线索(理由见 `errors.py` 的 `_classify_bad_request`)。解析不出这个形状时
|
||||
不静默,按状态码分类并把原始文本带上。
|
||||
"""
|
||||
if response.is_success:
|
||||
try:
|
||||
return response.json()
|
||||
except ValueError as exc:
|
||||
raise OpenAiError(
|
||||
classify(
|
||||
response.status_code,
|
||||
message=f"响应不是合法 JSON:{response.text[:200]}",
|
||||
),
|
||||
response.status_code,
|
||||
) from exc
|
||||
|
||||
error_type: str | None = None
|
||||
error_code: str | None = None
|
||||
error_param: str | None = None
|
||||
message = response.text[:500]
|
||||
try:
|
||||
error = response.json().get("error") or {}
|
||||
error_type = error.get("type")
|
||||
error_code = error.get("code")
|
||||
error_param = error.get("param")
|
||||
message = error.get("message") or message
|
||||
except ValueError:
|
||||
pass
|
||||
|
||||
failure = classify(
|
||||
response.status_code, error_type, error_code, message, error_param
|
||||
)
|
||||
log.warning(
|
||||
"OpenAI 返回错误 http=%s type=%s code=%s param=%s retry=%s",
|
||||
response.status_code,
|
||||
error_type,
|
||||
error_code,
|
||||
error_param,
|
||||
failure.retry.value,
|
||||
)
|
||||
raise OpenAiError(failure, response.status_code)
|
||||
|
||||
def close(self) -> None:
|
||||
# 只关自己创建的连接池:注入进来的归调用方管理。
|
||||
if self._owns_client:
|
||||
self._client.close()
|
||||
|
||||
def __enter__(self) -> Self:
|
||||
return self
|
||||
|
||||
def __exit__(self, *exc_info: object) -> None:
|
||||
self.close()
|
||||
@@ -0,0 +1,172 @@
|
||||
"""OpenAI 错误 → 领域 `Failure`。
|
||||
|
||||
映射依据:用户提供的 `doc_temp/gpt错误码.md`(官方 Error codes 页)。本模块曾经是
|
||||
按 HTTP 状态码推断的,拿到该文档后已按实际码名重写——**此前猜的五个码名
|
||||
(`rate_limit_exceeded`、`moderation_blocked`、`content_policy_violation`、
|
||||
`invalid_api_key`、`image_generation_user_error`)在官方文档里一个都不存在**,
|
||||
已全部删除。这段历史留在这里,作为「凭记忆列错误码表」代价的记录。
|
||||
|
||||
## ⚠️ 官方错误码表里没有任何业务层失败
|
||||
|
||||
文档的 13 条全是**基础设施与计费**:鉴权、IP/地域、限流、余额与限额、服务过载。
|
||||
**没有内容审核、没有版权/侵权、没有「这个提示词不能生成」。** 但图片接口明确有
|
||||
`moderation` 参数(`low`/`auto`),说明审核确实存在——只是它的错误形态未被文档化。
|
||||
|
||||
这对我们的直接后果:**一次侵权或违规拒绝很可能以一个普通 400 到达**,只带一句
|
||||
人类可读的消息。若把这种 400 一律归为「服务端错误」,用户会看到「服务出错了」
|
||||
然后反复重试同一个注定被拒的提示词。故本模块用 `error.param` 做结构化区分,
|
||||
见 `_classify_bad_request()`。那是**推断,不是文档事实**,已就地标注。
|
||||
|
||||
## 未处理的一项:`Retry-After`
|
||||
|
||||
文档在 429 与 503 处反复要求遵守 `Retry-After` 响应头。本服务目前**不读它**,
|
||||
理由是 Java 侧 Worker 的退避阶梯基线已是 30 秒、指数增长、上限 10 分钟,**在绝大
|
||||
多数情形下已长于服务商给的 `Retry-After`**——也就是我们已经比要求更保守,忽略它
|
||||
只会让重试偏慢,不会加剧限流。要把它接起来是一次跨语言契约变更(领域 `Failure`
|
||||
加字段 + `openapi-internal.yaml` 小版本 + Java 侧 DTO 与调度),收益只是延迟优化,
|
||||
故记在取证页的待办里而不是现在做。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
from app.domain.generation import Failure, Retry, UserFacing
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: 文档化的错误标识 → 领域处置。
|
||||
#:
|
||||
#: **按 `code` 与 `type` 两个字段分别查同一张表**,而不是各维护一张。理由:文档
|
||||
#: 把某个标识列在哪个字段下,并不保证实际响应里它一定出现在那个字段(`insufficient_quota`
|
||||
#: 就既被描述为「broader `error.type`」,又很容易出现在 `code` 里)。查错字段的代价
|
||||
#: 是把配额耗尽降级成限流、白烧一轮重试预算;而两边都查不引入任何**新**词汇——
|
||||
#: 表里每个键都抄自文档。
|
||||
_RULES: dict[str, tuple[Retry, UserFacing]] = {
|
||||
# ---- 计费/额度(官方表 429 一节,文档列在 `code` 下)----
|
||||
#
|
||||
# 这四条共同的性质是文档明说的那句:「Retrying billing, spend, or quota errors
|
||||
# won't restore API access」——重试不会恢复访问,要人去充值或改限额。这正是
|
||||
# `Retry.QUOTA` 存在的理由:Worker 不该把重试预算花在这上面,应直接终态失败
|
||||
# 并让用户看到出口。
|
||||
#
|
||||
# 预付余额耗尽。
|
||||
"credit_balance_exhausted": (Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED),
|
||||
# 组织月度消费限额(自己设的)。
|
||||
"organization_spend_limit_exceeded": (Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED),
|
||||
# 项目月度消费限额(自己设的)。
|
||||
"project_spend_limit_exceeded": (Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED),
|
||||
# OpenAI 分配的用量限额(不是自己设的,要申请提额)。
|
||||
"organization_usage_limit_exceeded": (Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED),
|
||||
# 文档原文:「The broader `error.type` can still be `insufficient_quota`」
|
||||
# ——计费类错误的宽类型,具体原因要看 code。
|
||||
"insufficient_quota": (Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED),
|
||||
# ---- 限流 ----
|
||||
#
|
||||
# `slow_down` 是「涨得太快」而非「超了额定速率」——文档说即使在 RPM/TPM 限额
|
||||
# 之内也会触发。两者处置相同(退避后重试),故不再细分。
|
||||
"slow_down": (Retry.BACKOFF, UserFacing.RATE_LIMITED),
|
||||
"rate_limit_error": (Retry.BACKOFF, UserFacing.RATE_LIMITED),
|
||||
# ---- 服务过载 ----
|
||||
#
|
||||
# 文档特别提醒:**视频过载已从 429 改判为 503**,且 Python SDK 分别抛
|
||||
# RateLimitError 与 InternalServerError,只捕一类会漏。我们两者都归可重试,
|
||||
# 故这次厂商侧的改判对本服务无影响——记下来是为了说明「为什么我们没被影响」。
|
||||
"server_is_overloaded": (Retry.BACKOFF, UserFacing.UPSTREAM_UNAVAILABLE),
|
||||
"service_unavailable_error": (Retry.BACKOFF, UserFacing.UPSTREAM_UNAVAILABLE),
|
||||
}
|
||||
|
||||
|
||||
def classify(
|
||||
http_status: int | None,
|
||||
error_type: str | None = None,
|
||||
error_code: str | None = None,
|
||||
message: str | None = None,
|
||||
error_param: str | None = None,
|
||||
) -> Failure:
|
||||
"""把一次失败分类。
|
||||
|
||||
判定顺序:`code` → `type` → HTTP 状态码。越具体的依据优先——`code` 是文档明确
|
||||
列出取值的字段,状态码只是最后的兜底。
|
||||
|
||||
:param http_status: `None` 表示网络层失败(连不上、超时),按可重试处理。
|
||||
:param error_param: 厂商指出的出错参数名。**它的有无是我们区分「参数错」与
|
||||
「内容被拒」的唯一结构化线索**,见 `_classify_bad_request()`。
|
||||
"""
|
||||
text = message or ""
|
||||
# provider_code 永远不为空——它是排查时的第一线索。
|
||||
provider_code = error_code or error_type or _status_label(http_status)
|
||||
|
||||
# code 优先于 type:文档里 code 比 type 具体(同一个 429 下六种 code 共享
|
||||
# 少数几个 type)。两者都查同一张表,理由见 `_RULES`。
|
||||
rule = _RULES.get(error_code or "") or _RULES.get(error_type or "")
|
||||
if rule:
|
||||
retry, user_facing = rule
|
||||
return Failure(retry, user_facing, provider_code, text)
|
||||
|
||||
if http_status is None:
|
||||
return Failure(
|
||||
Retry.BACKOFF, UserFacing.UPSTREAM_UNAVAILABLE, provider_code, text
|
||||
)
|
||||
|
||||
if http_status == 429:
|
||||
# 到这里说明既没有 code 也没有认识的 type。官方表里 429 的六种中,只有
|
||||
# 「Rate limit reached for requests」没有文档化的 code——故无码的 429
|
||||
# **按限流处理是文档支持的默认**,不再是一个猜测。
|
||||
return Failure(Retry.BACKOFF, UserFacing.RATE_LIMITED, provider_code, text)
|
||||
|
||||
if http_status == 400:
|
||||
return _classify_bad_request(provider_code, text, error_param)
|
||||
|
||||
if http_status in (401, 403):
|
||||
# 401:密钥无效/过期/不属于该组织;403:地域不受支持。
|
||||
# 两者都是**我方配置问题**,重试无用,且都不该把锅甩给用户的提示词。
|
||||
return Failure(Retry.NEVER, UserFacing.SERVER_ERROR, provider_code, text)
|
||||
|
||||
if 400 < http_status < 500:
|
||||
return Failure(Retry.NEVER, UserFacing.SERVER_ERROR, provider_code, text)
|
||||
|
||||
# 5xx 与一切未知:可重试。把一次可恢复的抖动判死是更糟的错误,而重试预算
|
||||
# 本身有上限(max_attempts),故偏向重试是有界的。
|
||||
return Failure(Retry.BACKOFF, UserFacing.UPSTREAM_UNAVAILABLE, provider_code, text)
|
||||
|
||||
|
||||
def _classify_bad_request(
|
||||
provider_code: str, text: str, error_param: str | None
|
||||
) -> Failure:
|
||||
"""400 的两义区分:参数错 vs 内容被拒。
|
||||
|
||||
**这是本模块唯一的推断,不是文档事实。**
|
||||
|
||||
依据与理由:
|
||||
|
||||
- 文档在 `service_tier` 那条明确写了「as an `invalid_request_error` with
|
||||
`error.param` set to `service_tier`」——即**参数类错误会带 `param`**。
|
||||
- 官方错误码表**没有任何内容审核/侵权码**,故一次内容拒绝只能以某种通用
|
||||
400 到达,且它不指向任何具体参数,`param` 应为空。
|
||||
- 我方在调用前已用 `capabilities.py` 校验过尺寸、画质、格式,`model`/`n` 由
|
||||
程序生成。**能活着到达服务商并被 400 拒掉的,参数嫌疑本来就不大。**
|
||||
|
||||
故:带 `param` → 参数错(服务端错误);不带 `param` → 按内容被拒处理。
|
||||
|
||||
取错的代价是不对称的,这也是选这个方向的理由:把内容拒绝报成「服务出错」会
|
||||
让用户反复重试同一个注定被拒的提示词;反过来把一个参数 bug 报成「内容可能不
|
||||
被支持」只是让用户多试一次,而排查者看到的 `provider_code` 与日志仍是准确的。
|
||||
|
||||
拿到真实的审核错误形态后,回来把这里换成确定的判定并更新取证页。
|
||||
"""
|
||||
if error_param:
|
||||
return Failure(Retry.NEVER, UserFacing.SERVER_ERROR, provider_code, text)
|
||||
|
||||
# 记 warning 而不是静默:这条分支是推断出来的,日志里能看到它被走了多少次、
|
||||
# 携带的是什么消息,是日后校正它的唯一素材来源。
|
||||
log.warning(
|
||||
"400 且无 error.param,按内容被拒处理(推断,非文档事实):code=%s message=%s",
|
||||
provider_code,
|
||||
text[:200],
|
||||
)
|
||||
return Failure(Retry.NEVER, UserFacing.CONTENT_REJECTED, provider_code, text)
|
||||
|
||||
|
||||
def _status_label(http_status: int | None) -> str:
|
||||
return f"HTTP_{http_status}" if http_status else "NETWORK_ERROR"
|
||||
@@ -0,0 +1,253 @@
|
||||
"""gpt-image-2 图片生成:文生图与图生图(图像编辑)。
|
||||
|
||||
## 与火山视频适配器的三处根本差异
|
||||
|
||||
1. **接口是同步的。** `POST /images/generations` 一次请求等到底,**没有任务 ID、
|
||||
没有查询接口**。差异由 `tasks.SyncTaskRegistry` 在本服务内部吸收(ADR-023 §4),
|
||||
代价与风险见该模块顶部说明。
|
||||
2. **产物是 base64 字节,不是 URL。** GPT image 系列**始终**返回 base64
|
||||
(`response_format` 对它们不支持)。好处是没有「URL 24 小时过期」问题;坏处是
|
||||
字节只在内存里,必须立刻转存——而转存阻塞于存储决策(ADR-023 §7)。
|
||||
3. **图生图走 multipart,不是 JSON。** `/images/edits` 的 `image` 字段是文件。
|
||||
我们的输入是一个 URL,故需先把它下载下来再作为 multipart 上传。
|
||||
|
||||
## 尺寸从响应直接拿,不需要对照表
|
||||
|
||||
响应带 `size` 字段(`WIDTHxHEIGHT`),故像素宽高是确定值——不像火山那边只给
|
||||
`ratio`+`resolution`、需要一张随模型族变化的对照表。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from app.domain.generation import (
|
||||
Failure,
|
||||
GenerationRequest,
|
||||
GenerationResult,
|
||||
MediaKind,
|
||||
Retry,
|
||||
TaskState,
|
||||
UserFacing,
|
||||
)
|
||||
from app.providers.base import GenerationProvider
|
||||
|
||||
from .capabilities import (
|
||||
parse_size,
|
||||
validate_output_format,
|
||||
validate_quality,
|
||||
validate_size,
|
||||
)
|
||||
from .client import OpenAiClient
|
||||
from .models import ImagesResponse, ImageSubmission
|
||||
from .tasks import SyncTaskRegistry
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: `creation.generation_models.provider_code` 里对应本适配器的值。
|
||||
PROVIDER_CODE = "openai-gpt-image"
|
||||
|
||||
#: 下载参考图的超时。这是一次内部/公网取图,与生成耗时无关,故很短。
|
||||
_INPUT_FETCH_TIMEOUT_SEC = 15.0
|
||||
|
||||
#: 参考图大小上限。服务商限制单图 < 50MB;这里取更小的值,因为我们还要把它整个
|
||||
#: 读进内存再转发——一张几十 MB 的图会让内存占用随并发线性上涨。
|
||||
_MAX_INPUT_BYTES = 20 * 1024 * 1024
|
||||
|
||||
|
||||
class OpenAiImageProvider(GenerationProvider):
|
||||
"""gpt-image-2 的图片生成适配器。"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
client: OpenAiClient,
|
||||
registry: SyncTaskRegistry,
|
||||
default_model: str = "",
|
||||
input_fetch_client: httpx.Client | None = None,
|
||||
):
|
||||
self._client = client
|
||||
self._registry = registry
|
||||
self._default_model = default_model
|
||||
self._owns_fetch_client = input_fetch_client is None
|
||||
self._fetch_client = input_fetch_client or httpx.Client(
|
||||
timeout=_INPUT_FETCH_TIMEOUT_SEC, follow_redirects=True
|
||||
)
|
||||
|
||||
@property
|
||||
def provider_code(self) -> str:
|
||||
return PROVIDER_CODE
|
||||
|
||||
@property
|
||||
def media_kind(self) -> MediaKind:
|
||||
return MediaKind.IMAGE
|
||||
|
||||
# ------------------------------------------------------------- submit
|
||||
|
||||
def submit(self, request: GenerationRequest) -> str:
|
||||
if request.media_kind is not MediaKind.IMAGE:
|
||||
raise ValueError(
|
||||
f"{PROVIDER_CODE} 只处理图片生成,收到 {request.media_kind.value}"
|
||||
)
|
||||
|
||||
submission = self._to_submission(request)
|
||||
# 参数校验放在提交进注册表**之前**:违规请求应当立刻以 400 回给调用方,
|
||||
# 而不是变成一个注定失败的后台任务——后者要等一轮轮询才被发现,还占了一个
|
||||
# 线程池槽位。
|
||||
validate_size(submission.size)
|
||||
validate_quality(submission.quality)
|
||||
validate_output_format(
|
||||
submission.output_format, transparent=submission.background == "transparent"
|
||||
)
|
||||
|
||||
if request.input_url:
|
||||
# 图生图:先取图。**取图也放在受理之前**——取不到就是参数问题
|
||||
# (URL 不可达),应当立刻拒绝而不是变成一个后台失败。
|
||||
image_bytes, filename, content_type = self._fetch_input(request.input_url)
|
||||
work = lambda: self._run_edit(
|
||||
submission, image_bytes, filename, content_type
|
||||
)
|
||||
else:
|
||||
work = lambda: self._run_generate(submission)
|
||||
|
||||
# 注册表满时 `submit` 抛 RuntimeError,这里**刻意不接**:归成 ValueError 让上层
|
||||
# 映射成 invalid_request 是错的——那不是请求的问题而是容量问题,且稍后重试会
|
||||
# 好。原样上抛,由端点映射为 provider_unavailable(503)。
|
||||
return self._registry.submit(work)
|
||||
|
||||
def _to_submission(self, request: GenerationRequest) -> ImageSubmission:
|
||||
"""领域请求 → 厂商请求体。
|
||||
|
||||
模型优先取请求里的(来自任务行冻结的快照),仅在缺失时回落到配置的默认值
|
||||
——适配层不自行选模型,且任务行的快照必须与实际调用一致,否则无法对账。
|
||||
"""
|
||||
model = request.model or self._default_model
|
||||
if not model:
|
||||
raise ValueError(
|
||||
"未指定模型:请求未携带 model 且 OPENAI_IMAGE_MODEL 未配置"
|
||||
)
|
||||
|
||||
return ImageSubmission(
|
||||
model=model,
|
||||
prompt=request.prompt or "",
|
||||
# 领域层的 ratio/resolution 是为视频设计的;图片用像素尺寸。宽高都有
|
||||
# 才拼 size,否则交给模型自选(auto)——半个尺寸不如不给。
|
||||
size=(
|
||||
f"{request.width_px}x{request.height_px}"
|
||||
if request.width_px and request.height_px
|
||||
else None
|
||||
),
|
||||
safety_identifier=request.safety_identifier,
|
||||
)
|
||||
|
||||
# -------------------------------------------------------------- fetch
|
||||
|
||||
def fetch(self, provider_request_id: str) -> GenerationResult:
|
||||
result = self._registry.fetch(provider_request_id)
|
||||
if result is None:
|
||||
# 未知 ID:从未存在过,或进程重启后丢失(两者无法区分,见 tasks.py)。
|
||||
# 抛 KeyError 让端点映射成 task_not_found——调用方要做的是停止轮询,
|
||||
# 而不是退避重试。
|
||||
raise KeyError(provider_request_id)
|
||||
return result
|
||||
|
||||
# ------------------------------------------------- 实际的同步调用
|
||||
|
||||
def _run_generate(self, submission: ImageSubmission) -> GenerationResult:
|
||||
"""文生图。在注册表的后台线程里执行。"""
|
||||
payload = self._client.post_json(
|
||||
"/images/generations", submission.to_json_body()
|
||||
)
|
||||
return self._to_result(ImagesResponse.from_payload(payload))
|
||||
|
||||
def _run_edit(
|
||||
self,
|
||||
submission: ImageSubmission,
|
||||
image_bytes: bytes,
|
||||
filename: str,
|
||||
content_type: str,
|
||||
) -> GenerationResult:
|
||||
"""图生图(图像编辑)。走 multipart。"""
|
||||
payload = self._client.post_multipart(
|
||||
"/images/edits",
|
||||
data=submission.to_form_fields(),
|
||||
# 字段名用 `image[]`:编辑接口支持多图且共用该字段名。单图时也用它,
|
||||
# 避免"单图一种写法、多图另一种"的分叉。
|
||||
files=[("image[]", (filename, image_bytes, content_type))],
|
||||
)
|
||||
return self._to_result(ImagesResponse.from_payload(payload))
|
||||
|
||||
def _to_result(self, response: ImagesResponse) -> GenerationResult:
|
||||
"""厂商响应 → 领域结果。"""
|
||||
image = response.first_image()
|
||||
if image is None or not image.b64_json:
|
||||
# 2xx 却没有图片数据。当成可重试的上游异常,而不是把一个没有产物的
|
||||
# 「成功」放给上层去空指针。
|
||||
log.error("OpenAI 返回 2xx 但缺少图片数据:%r", response.raw_keys())
|
||||
return GenerationResult(
|
||||
provider_request_id="",
|
||||
state=TaskState.FAILED,
|
||||
failure=Failure(
|
||||
Retry.BACKOFF,
|
||||
UserFacing.UPSTREAM_UNAVAILABLE,
|
||||
"MissingImageData",
|
||||
"服务商返回成功但响应中没有图片数据",
|
||||
),
|
||||
)
|
||||
|
||||
size = parse_size(response.size or "")
|
||||
return GenerationResult(
|
||||
# 真实 ID 由注册表在 fetch 时填入——这里填不了,因为工作函数运行时
|
||||
# 还拿不到自己的任务 ID。
|
||||
provider_request_id="",
|
||||
state=TaskState.SUCCEEDED,
|
||||
progress=100,
|
||||
# ⚠️ 产物是 base64 字节而非可访问 URL。这里放 data URI 只是为了让领域
|
||||
# 模型的「SUCCEEDED 必须有 asset_url」不变式成立,**它不是一个可长期
|
||||
# 保存的地址**:一张 1024×1024 的 png 约 1~2MB,base64 后更大,绝不
|
||||
# 应该原样写进 media.assets.external_url。转存落地后这里应改为返回
|
||||
# 上传后的 object key。
|
||||
asset_url=f"data:image/{response.output_format or 'png'};base64,"
|
||||
+ image.b64_json,
|
||||
# 无时效——字节在手,不像火山 URL 会过期。用 None 而非一个假的秒数。
|
||||
asset_url_expires_in_sec=None,
|
||||
width_px=size.width if size else None,
|
||||
height_px=size.height if size else None,
|
||||
output_format=response.output_format,
|
||||
billing_tokens=response.total_tokens,
|
||||
)
|
||||
|
||||
# ------------------------------------------------------------ helpers
|
||||
|
||||
def _fetch_input(self, url: str) -> tuple[bytes, str, str]:
|
||||
"""下载参考图,供 multipart 上传。
|
||||
|
||||
服务商要求 png / webp / jpg 且单图 < 50MB。这里额外设了更小的上限,因为
|
||||
字节要整个进内存再转发。
|
||||
"""
|
||||
try:
|
||||
with self._fetch_client.stream("GET", url) as response:
|
||||
response.raise_for_status()
|
||||
declared = response.headers.get("Content-Length")
|
||||
if declared and int(declared) > _MAX_INPUT_BYTES:
|
||||
raise ValueError(
|
||||
f"参考图过大({declared} 字节),上限 {_MAX_INPUT_BYTES}"
|
||||
)
|
||||
chunks: list[bytes] = []
|
||||
total = 0
|
||||
for chunk in response.iter_bytes():
|
||||
total += len(chunk)
|
||||
if total > _MAX_INPUT_BYTES:
|
||||
# 不信 Content-Length:它可能缺失或撒谎,边读边数才拦得住。
|
||||
raise ValueError(f"参考图超过上限 {_MAX_INPUT_BYTES} 字节")
|
||||
chunks.append(chunk)
|
||||
content_type = response.headers.get("Content-Type", "image/png")
|
||||
except httpx.HTTPError as exc:
|
||||
raise ValueError(f"参考图无法下载:{url}({exc})") from exc
|
||||
|
||||
return b"".join(chunks), "input.png", content_type.split(";")[0]
|
||||
|
||||
def close(self) -> None:
|
||||
if self._owns_fetch_client:
|
||||
self._fetch_client.close()
|
||||
@@ -0,0 +1,149 @@
|
||||
"""OpenAI 图片接口的请求/响应 DTO。
|
||||
|
||||
只放**厂商词汇**。领域概念在 `app/domain/generation.py`,翻译在 `image.py`。
|
||||
|
||||
一处值得注意的厂商差异:同一份请求体在**文生图走 JSON、图生图走 multipart**
|
||||
(后者的 `image` 是文件)。故本类提供两种输出形态——`to_json_body()` 与
|
||||
`to_form_fields()`。multipart 的字段值必须是字符串,布尔与数字要转,不能直接塞。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
from typing import Any
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImageSubmission:
|
||||
"""一次图片生成请求。
|
||||
|
||||
刻意只暴露 Patbond 会用到的参数。厂商参数远多于此(`n`、`partial_images`、
|
||||
`stream`、`mask`、`input_fidelity`、`output_compression`、`moderation`…),
|
||||
多暴露一个就多一处要在契约、客户端、文档三处同步维护的地方。
|
||||
|
||||
:param n: **固定为 1,不开放**。领域模型的一个任务对应一个产物
|
||||
(`generation_jobs.output_asset_id` 是单值);允许 n>1 就得改数据模型。
|
||||
:param quality: `None` 交给服务商默认(auto)。注意 `xhigh`/`max` 仅
|
||||
gpt-image-2.5 系列支持,gpt-image-2 传了会被拒(`capabilities` 会先拦)。
|
||||
:param background: `transparent` 时 `output_format` 只能 png/webp。
|
||||
文档称 gpt-image-2 对透明背景的支持**处于预览**,故默认不指定。
|
||||
:param safety_identifier: 对应厂商的 `user` 字段——终端用户标识,供其做滥用
|
||||
检测。必须是哈希,不可传明文用户 ID。
|
||||
"""
|
||||
|
||||
model: str
|
||||
prompt: str
|
||||
size: str | None = None
|
||||
quality: str | None = None
|
||||
output_format: str | None = None
|
||||
background: str | None = None
|
||||
safety_identifier: str | None = None
|
||||
|
||||
def to_json_body(self) -> dict[str, Any]:
|
||||
"""文生图(`/images/generations`)的 JSON 体。"""
|
||||
body: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"prompt": self.prompt,
|
||||
# 一个任务一个产物,见类 docstring。
|
||||
"n": 1,
|
||||
}
|
||||
self._add_optional(body)
|
||||
return body
|
||||
|
||||
def to_form_fields(self) -> dict[str, str]:
|
||||
"""图生图(`/images/edits`)的 multipart 文本字段。
|
||||
|
||||
multipart 的字段值必须是字符串——数字与布尔直接塞进去会被 httpx 拒绝或
|
||||
编码成意外形态。
|
||||
"""
|
||||
fields: dict[str, Any] = {
|
||||
"model": self.model,
|
||||
"prompt": self.prompt,
|
||||
"n": 1,
|
||||
}
|
||||
self._add_optional(fields)
|
||||
return {key: str(value) for key, value in fields.items()}
|
||||
|
||||
def _add_optional(self, target: dict[str, Any]) -> None:
|
||||
"""只加已设置的可选项。
|
||||
|
||||
不设的字段**留空而不显式传 null**:服务商对每个参数都有文档化的默认值,
|
||||
显式 null 可能覆盖掉那些默认值。
|
||||
"""
|
||||
if self.size:
|
||||
target["size"] = self.size
|
||||
if self.quality:
|
||||
target["quality"] = self.quality
|
||||
if self.output_format:
|
||||
target["output_format"] = self.output_format
|
||||
if self.background:
|
||||
target["background"] = self.background
|
||||
if self.safety_identifier:
|
||||
# 厂商字段名是 `user`,不是 safety_identifier。
|
||||
target["user"] = self.safety_identifier
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class GeneratedImage:
|
||||
"""`data[]` 里的一张图。
|
||||
|
||||
:param b64_json: base64 编码的图片。**GPT image 系列始终返回它**
|
||||
(`response_format` 对这些模型不支持,`url` 只用于 dall-e)。
|
||||
"""
|
||||
|
||||
b64_json: str | None = None
|
||||
url: str | None = None
|
||||
revised_prompt: str | None = None
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ImagesResponse:
|
||||
"""`/images/generations` 与 `/images/edits` 的共同响应形状。
|
||||
|
||||
:param size: `WIDTHxHEIGHT` 字符串。**像素宽高由此直接得到**,不像火山那边
|
||||
只给 ratio+resolution、需要一张随模型族变化的对照表。
|
||||
:param total_tokens: 计费对账依据(`usage.total_tokens`)。
|
||||
"""
|
||||
|
||||
created: int | None = None
|
||||
data: tuple[GeneratedImage, ...] = ()
|
||||
size: str | None = None
|
||||
output_format: str | None = None
|
||||
quality: str | None = None
|
||||
background: str | None = None
|
||||
total_tokens: int | None = None
|
||||
input_tokens: int | None = None
|
||||
output_tokens: int | None = None
|
||||
_raw: dict[str, Any] | None = None
|
||||
|
||||
def first_image(self) -> GeneratedImage | None:
|
||||
return self.data[0] if self.data else None
|
||||
|
||||
def raw_keys(self) -> list[str]:
|
||||
"""原始响应的顶层键,仅用于「2xx 但没有图」时的日志。"""
|
||||
return sorted(self._raw) if self._raw else []
|
||||
|
||||
@classmethod
|
||||
def from_payload(cls, payload: dict[str, Any]) -> ImagesResponse:
|
||||
usage = payload.get("usage") or {}
|
||||
images = tuple(
|
||||
GeneratedImage(
|
||||
b64_json=item.get("b64_json"),
|
||||
url=item.get("url"),
|
||||
revised_prompt=item.get("revised_prompt"),
|
||||
)
|
||||
for item in (payload.get("data") or [])
|
||||
if isinstance(item, dict)
|
||||
)
|
||||
return cls(
|
||||
created=payload.get("created"),
|
||||
data=images,
|
||||
size=payload.get("size"),
|
||||
output_format=payload.get("output_format"),
|
||||
quality=payload.get("quality"),
|
||||
background=payload.get("background"),
|
||||
total_tokens=usage.get("total_tokens"),
|
||||
input_tokens=usage.get("input_tokens"),
|
||||
output_tokens=usage.get("output_tokens"),
|
||||
_raw=payload,
|
||||
)
|
||||
@@ -0,0 +1,224 @@
|
||||
"""同步型供应商的任务注册表:把「一次调用等到底」转成「提交 + 查询」。
|
||||
|
||||
## 为什么必须有它
|
||||
|
||||
ADR-023 §4 定的是「Java↔Python 边界一律异步,同步型供应商由其适配层在内部消化
|
||||
这个差异」。火山的视频接口本身异步,所以视频适配器不需要这一层。**OpenAI 的图片
|
||||
接口是同步的**——`POST /images/generations` 在一次请求里等到图片生成完才返回,
|
||||
没有任务 ID,也没有查询接口。差异必须在这里被吸收,否则就会渗透到客户端契约、
|
||||
等待态设计和 E2E,那是 §4 明确要避免的。
|
||||
|
||||
## 这与「Python 不持有状态」的张力,以及取舍
|
||||
|
||||
Seedance 那条路径上,Python 确实无状态:任务状态的唯一事实来源是 Java 的
|
||||
`generation_jobs`,Python 只是个转发器。**图片这条路径做不到**——同步接口没有可
|
||||
供事后查询的厂商 ID,所以「在途任务」这件事只能记在本进程内存里。
|
||||
|
||||
代价与处置如实登记:
|
||||
|
||||
1. **进程重启会丢失在途任务。** 之后 Java 轮询会得到 `task_not_found`,任务被判
|
||||
终态失败,而**那次调用可能已经完成并计费**。OpenAI 的图片接口没有任务 ID 可供
|
||||
事后查询,所以这个窗口**无法靠"再查一次"弥补**,只能由 `max_attempts` 兜住。
|
||||
这与火山那边「提交超时不知道有没有扣费」是同一类风险,但触发面更大(那边只有
|
||||
提交超时,这边还包括进程重启)。
|
||||
2. **产物是 base64 字节,只在内存里。** 完成的任务必须在 TTL 内被取走,否则丢失。
|
||||
而转存阻塞于对象存储决策(ADR-023 §7),所以**图片链路在存储决策落地前比视频
|
||||
更受限**:视频至少有个 24 小时的厂商 URL 可以先看效果,图片的字节一旦被清理
|
||||
就什么都没有了。
|
||||
3. 注册表**有界且有 TTL**,否则它就是个内存泄漏。
|
||||
|
||||
## 为什么用线程而不是 asyncio
|
||||
|
||||
FastAPI 的端点在本服务里是同步函数(`def` 而非 `async def`),且 `httpx.Client`
|
||||
是同步客户端。用线程池承载那个长调用与现有代码一致;改成 asyncio 需要把传输层、
|
||||
端点、测试全部改成异步,收益只是少几个线程——本服务的并发量级(一台轻量服务器上
|
||||
的少量生成请求)用不上那个收益。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
import threading
|
||||
import time
|
||||
import uuid
|
||||
from collections.abc import Callable
|
||||
from concurrent.futures import Future, ThreadPoolExecutor
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
from app.domain.generation import Failure, GenerationResult, TaskState
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
#: 本地任务 ID 的前缀,便于在日志里与厂商 ID 区分开。
|
||||
LOCAL_TASK_PREFIX = "local-"
|
||||
|
||||
|
||||
@dataclass
|
||||
class _Entry:
|
||||
"""一条在途或已完成的任务。"""
|
||||
|
||||
future: Future
|
||||
created_at: float = field(default_factory=time.monotonic)
|
||||
#: **首次观察到**完成的时刻;未完成为 None。TTL 从这里起算而不是从创建起算——
|
||||
#: 否则一个耗时接近 TTL 的任务会在刚完成时就被清理掉,结果还没被取走。
|
||||
#:
|
||||
#: 为什么是「首次观察到」而不是「实际完成」:实际完成时刻需要一个
|
||||
#: `add_done_callback` 才能捕捉,而那会**死锁**——快速完成的任务在
|
||||
#: `add_done_callback` 时已经 done,回调便**同步地在调用线程执行**,而该线程
|
||||
#: 此刻正持有本注册表的锁(非可重入)。改用惰性记录既无此风险,又只会让结果
|
||||
#: 多留一会儿(保守方向),不会让它提前消失。
|
||||
finished_at: float | None = None
|
||||
|
||||
|
||||
class SyncTaskRegistry:
|
||||
"""把同步调用包装成可轮询的任务。
|
||||
|
||||
线程安全:`submit`/`fetch`/清理可能来自不同请求线程。
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
*,
|
||||
max_workers: int = 4,
|
||||
completed_ttl_sec: float = 900.0,
|
||||
max_entries: int = 256,
|
||||
):
|
||||
"""
|
||||
:param max_workers: 并发执行的同步调用数上限。**同时也是对服务商并发的
|
||||
自我限制**:超出的提交会排队而不是一起打过去,既避免把自己的 RPM 打爆,
|
||||
也让「同时有多少钱在花」有个上界。
|
||||
:param completed_ttl_sec: 已完成任务的保留时长。需大于 Java 侧最长的轮询
|
||||
间隔(当前阶梯最长 30 秒),否则结果会在被取走之前被清理。默认 15
|
||||
分钟留足余量。
|
||||
:param max_entries: 注册表容量上限。满了先清理过期项,仍满则拒绝新提交
|
||||
(而不是无声挤掉某个在途任务——那会让一个已经在计费的调用变成孤儿)。
|
||||
"""
|
||||
self._executor = ThreadPoolExecutor(
|
||||
max_workers=max_workers, thread_name_prefix="sync-gen"
|
||||
)
|
||||
self._completed_ttl_sec = completed_ttl_sec
|
||||
self._max_entries = max_entries
|
||||
self._entries: dict[str, _Entry] = {}
|
||||
self._lock = threading.Lock()
|
||||
|
||||
def submit(self, work: Callable[[], GenerationResult]) -> str:
|
||||
"""安排一次同步调用在后台执行,立刻返回本地任务 ID。
|
||||
|
||||
:raises RuntimeError: 注册表已满(并发在途任务过多)
|
||||
"""
|
||||
with self._lock:
|
||||
self._evict_expired_locked()
|
||||
if len(self._entries) >= self._max_entries:
|
||||
raise RuntimeError(
|
||||
f"在途生成任务已达上限 {self._max_entries},请稍后重试"
|
||||
)
|
||||
task_id = LOCAL_TASK_PREFIX + uuid.uuid4().hex
|
||||
# 刻意不注册 done_callback,原因见 _Entry.finished_at 的说明(会死锁)。
|
||||
self._entries[task_id] = _Entry(future=self._executor.submit(work))
|
||||
log.info("已受理同步生成任务 %s(在途 %d)", task_id, len(self._entries))
|
||||
return task_id
|
||||
|
||||
def fetch(self, task_id: str) -> GenerationResult | None:
|
||||
"""查任务;未知 ID 返回 None。
|
||||
|
||||
未知 ID 的两种成因**无法区分**:从未存在过,或进程重启后丢失。两者对调用
|
||||
方的处置相同(停止轮询、任务判失败),故不强行区分——区分不了的事不要假装
|
||||
区分。
|
||||
"""
|
||||
with self._lock:
|
||||
entry = self._entries.get(task_id)
|
||||
if entry is not None and entry.finished_at is None and entry.future.done():
|
||||
# 首次观察到完成:记下时刻,TTL 由此起算。
|
||||
entry.finished_at = time.monotonic()
|
||||
if entry is None:
|
||||
return None
|
||||
|
||||
if not entry.future.done():
|
||||
# 同步接口不提供进度,故只能报 0。客户端契约本就不承诺平滑进度
|
||||
# (ADR-023 §10),一个诚实的 0 不该被读成「卡住」。
|
||||
return GenerationResult(
|
||||
provider_request_id=task_id, state=TaskState.RUNNING
|
||||
)
|
||||
|
||||
try:
|
||||
result = entry.future.result()
|
||||
except Exception as exc: # noqa: BLE001 - 兜底转成领域失败
|
||||
return self._failure_result(task_id, exc)
|
||||
# 用真实的任务 ID 覆盖工作函数里可能留空的那个字段。
|
||||
return GenerationResult(
|
||||
provider_request_id=task_id,
|
||||
state=result.state,
|
||||
progress=result.progress,
|
||||
asset_url=result.asset_url,
|
||||
asset_url_expires_in_sec=result.asset_url_expires_in_sec,
|
||||
cover_url=result.cover_url,
|
||||
ratio=result.ratio,
|
||||
resolution=result.resolution,
|
||||
width_px=result.width_px,
|
||||
height_px=result.height_px,
|
||||
duration_ms=result.duration_ms,
|
||||
output_format=result.output_format,
|
||||
failure=result.failure,
|
||||
billing_tokens=result.billing_tokens,
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _failure_result(task_id: str, exc: BaseException) -> GenerationResult:
|
||||
"""工作函数抛出的异常转成领域失败。
|
||||
|
||||
已分类的服务商异常保留其分类;其余按可重试的上游问题处理,并把类型名带进
|
||||
`provider_code`——否则排查时只有一句空洞的错误消息。
|
||||
"""
|
||||
from app.domain.generation import Retry, UserFacing
|
||||
|
||||
from .client import OpenAiError
|
||||
|
||||
if isinstance(exc, OpenAiError):
|
||||
failure = exc.failure
|
||||
else:
|
||||
log.exception("同步生成任务 %s 抛出未分类异常", task_id)
|
||||
failure = Failure(
|
||||
Retry.BACKOFF,
|
||||
UserFacing.UPSTREAM_UNAVAILABLE,
|
||||
type(exc).__name__,
|
||||
str(exc),
|
||||
)
|
||||
return GenerationResult(
|
||||
provider_request_id=task_id, state=TaskState.FAILED, failure=failure
|
||||
)
|
||||
|
||||
def _evict_expired_locked(self) -> None:
|
||||
"""清理已完成且超过 TTL 的条目。调用方必须已持锁。
|
||||
|
||||
**只清已完成的**:在途任务无论多久都不清——它可能正在花钱,清掉会让它
|
||||
变成一个无人认领的开销。真正卡死的在途任务由线程池的上限兜住。
|
||||
"""
|
||||
now = time.monotonic()
|
||||
# 顺带给「已完成但从未被 fetch 过」的条目补上观察时刻,否则它们的
|
||||
# finished_at 永远是 None,就永远不会过期——那正是内存泄漏。
|
||||
for entry in self._entries.values():
|
||||
if entry.finished_at is None and entry.future.done():
|
||||
entry.finished_at = now
|
||||
expired = [
|
||||
task_id
|
||||
for task_id, entry in self._entries.items()
|
||||
if entry.finished_at is not None
|
||||
and now - entry.finished_at > self._completed_ttl_sec
|
||||
]
|
||||
for task_id in expired:
|
||||
del self._entries[task_id]
|
||||
if expired:
|
||||
log.debug("清理了 %d 个已完成的同步任务条目", len(expired))
|
||||
|
||||
@property
|
||||
def size(self) -> int:
|
||||
with self._lock:
|
||||
return len(self._entries)
|
||||
|
||||
def shutdown(self) -> None:
|
||||
"""停止接受新任务。
|
||||
|
||||
`wait=False`:在途的同步调用可能还要跑几分钟,进程退出时不等它们——结果
|
||||
反正已经无处可交付了。它们会随进程一起消失,这正是上面第 1 点登记的风险。
|
||||
"""
|
||||
self._executor.shutdown(wait=False, cancel_futures=True)
|
||||
@@ -34,6 +34,13 @@ class ProviderRegistry:
|
||||
"""
|
||||
return self._by_code.get(provider_code)
|
||||
|
||||
def all(self) -> list[GenerationProvider]:
|
||||
"""全部已装配的适配器。
|
||||
|
||||
供「按 media_kind 找适配器」这类查询用——按能力而非按 code 检索时用它。
|
||||
"""
|
||||
return list(self._by_code.values())
|
||||
|
||||
@property
|
||||
def codes(self) -> frozenset[str]:
|
||||
return frozenset(self._by_code)
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""各 Seedance 模型的参数限制。
|
||||
|
||||
**为什么需要它**:已接入的两个模型限制不同——2.5 时长上限 30 秒但没有 4k,
|
||||
2.0 有 4k 但时长上限 15 秒;2.5 在图生视频时 `ratio` 只能是 `adaptive`。超范围的
|
||||
请求会被厂商以 400 拒绝,那要花一次往返、计入 RPM,并表现为一个失败的任务。
|
||||
在本地先拦下来,错误信息还能具体指出是哪个模型的哪条限制。
|
||||
|
||||
**未知模型一律放行**(只记一条日志):账号新开通一个模型时,不应该因为这张表
|
||||
没更新就用不了——那会让「开通即可用」变成「开通还得改代码发版」。代价只是失去
|
||||
本地预校验,厂商仍会兜底。
|
||||
|
||||
限制取自官方文档,取证见 patbond-doc
|
||||
`docs/development/providers/volcengine-seedance.md` §5。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
|
||||
log = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ModelLimits:
|
||||
"""一个模型族的参数限制。
|
||||
|
||||
:param ratio_must_be_adaptive_with_first_frame: 2.5 在图生视频(首帧/首尾帧)
|
||||
与视频编辑/延长任务中**只支持 `adaptive`**——模型自动保持输出宽高比与
|
||||
首帧图片一致,不接受指定值。2.0 系列则可指定。
|
||||
:param supports_output_format: `output_format`(mp4/mov)仅 2.5 支持。
|
||||
"""
|
||||
|
||||
resolutions: frozenset[str]
|
||||
min_duration_sec: int
|
||||
max_duration_sec: int
|
||||
ratio_must_be_adaptive_with_first_frame: bool
|
||||
supports_output_format: bool
|
||||
|
||||
|
||||
#: Seedance 2.5:时长可到 30 秒,但没有 4k;图生视频时宽高比锁 adaptive。
|
||||
#: 1080p 输出为 10bit + H.265/HEVC,少数播放环境可能不兼容。
|
||||
SEEDANCE_2_5 = ModelLimits(
|
||||
resolutions=frozenset({"480p", "720p", "1080p"}),
|
||||
min_duration_sec=4,
|
||||
max_duration_sec=30,
|
||||
ratio_must_be_adaptive_with_first_frame=True,
|
||||
supports_output_format=True,
|
||||
)
|
||||
|
||||
#: Seedance 2.0 系列:有 4k,但时长上限 15 秒。4k 输出为 10bit + H.265/HEVC。
|
||||
SEEDANCE_2_0 = ModelLimits(
|
||||
resolutions=frozenset({"480p", "720p", "1080p", "4k"}),
|
||||
min_duration_sec=4,
|
||||
max_duration_sec=15,
|
||||
ratio_must_be_adaptive_with_first_frame=False,
|
||||
supports_output_format=False,
|
||||
)
|
||||
|
||||
#: 按模型族前缀匹配,而非精确匹配完整 model ID。
|
||||
#: 完整 ID 形如 `doubao-seedance-2-5-260628`,尾部的日期版本会随厂商更新而变;
|
||||
#: 按前缀匹配使一次模型小版本升级不需要改这张表。
|
||||
_BY_PREFIX: tuple[tuple[str, ModelLimits], ...] = (
|
||||
("doubao-seedance-2-5", SEEDANCE_2_5),
|
||||
("doubao-seedance-2-0", SEEDANCE_2_0),
|
||||
)
|
||||
|
||||
#: `duration` 的特殊值:交给模型在有效范围内自选。不参与范围校验。
|
||||
AUTO_DURATION = -1
|
||||
|
||||
|
||||
def limits_for(model: str) -> ModelLimits | None:
|
||||
"""查模型限制;未知模型返回 None(表示不做本地预校验)。"""
|
||||
for prefix, limits in _BY_PREFIX:
|
||||
if model.startswith(prefix):
|
||||
return limits
|
||||
return None
|
||||
|
||||
|
||||
def validate(
|
||||
model: str,
|
||||
*,
|
||||
resolution: str | None = None,
|
||||
duration_sec: int | None = None,
|
||||
ratio: str | None = None,
|
||||
has_first_frame: bool = False,
|
||||
) -> None:
|
||||
"""按模型限制校验参数,不合法则抛 `ValueError`。
|
||||
|
||||
未知模型直接放行——见模块 docstring。
|
||||
|
||||
:raises ValueError: 参数超出该模型的限制,消息里指明模型与具体限制
|
||||
"""
|
||||
limits = limits_for(model)
|
||||
if limits is None:
|
||||
log.info("模型 %s 无本地限制表,跳过预校验,由厂商兜底", model)
|
||||
return
|
||||
|
||||
if resolution and resolution not in limits.resolutions:
|
||||
raise ValueError(
|
||||
f"模型 {model} 不支持分辨率 {resolution},"
|
||||
f"可选:{sorted(limits.resolutions)}"
|
||||
)
|
||||
|
||||
if (
|
||||
duration_sec is not None
|
||||
and duration_sec != AUTO_DURATION
|
||||
and not limits.min_duration_sec <= duration_sec <= limits.max_duration_sec
|
||||
):
|
||||
raise ValueError(
|
||||
f"模型 {model} 的时长需在 "
|
||||
f"[{limits.min_duration_sec}, {limits.max_duration_sec}] 秒内"
|
||||
f"(或 {AUTO_DURATION} 表示由模型自选),当前 {duration_sec}"
|
||||
)
|
||||
|
||||
if (
|
||||
has_first_frame
|
||||
and limits.ratio_must_be_adaptive_with_first_frame
|
||||
and ratio
|
||||
and ratio != "adaptive"
|
||||
):
|
||||
raise ValueError(
|
||||
f"模型 {model} 在图生视频时只支持 ratio=adaptive"
|
||||
f"(输出宽高比自动与首帧图片一致),当前 {ratio}"
|
||||
)
|
||||
@@ -0,0 +1,85 @@
|
||||
"""`resolution` × `ratio` → 像素宽高的对照表。
|
||||
|
||||
**这是纯静态数据,不需要查询任何东西、也不依赖存储方案。** 厂商的查询接口会返回
|
||||
`ratio` 与 `resolution`(但不返回像素值),查本表即可换算——无需下载产物、无需
|
||||
探测文件。数值逐格取自官方文档「不同宽高比对应的宽高像素值」一节。
|
||||
|
||||
有了它,`widthPx`/`heightPx` 在 AI 视频产物上就能填真实值,而不必等到把文件
|
||||
下载下来解码——这也正是 ADR-023 §11 所说「AI 输出路径不需要解码」的落地方式。
|
||||
|
||||
**实现上的一个关键观察**:Seedance 2.5 与 2.0 系列的 24 格里**只有两格不同**
|
||||
(480p 的 `16:9` 与 `9:16`),其余完全一致。故实现为「共享表 + 两格覆盖」,
|
||||
而不是抄两份 24 行——后者每次改动都要改两处,迟早只改一处。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from .capabilities import SEEDANCE_2_0, SEEDANCE_2_5, ModelLimits
|
||||
|
||||
#: (resolution, ratio) → (width, height)。两个模型族共用的 22 格。
|
||||
_SHARED: dict[tuple[str, str], tuple[int, int]] = {
|
||||
("480p", "4:3"): (752, 560),
|
||||
("480p", "1:1"): (640, 640),
|
||||
("480p", "3:4"): (560, 752),
|
||||
("480p", "21:9"): (992, 432),
|
||||
("720p", "16:9"): (1280, 720),
|
||||
("720p", "4:3"): (1112, 834),
|
||||
("720p", "1:1"): (960, 960),
|
||||
("720p", "3:4"): (834, 1112),
|
||||
("720p", "9:16"): (720, 1280),
|
||||
("720p", "21:9"): (1470, 630),
|
||||
("1080p", "16:9"): (1920, 1080),
|
||||
("1080p", "4:3"): (1664, 1248),
|
||||
("1080p", "1:1"): (1440, 1440),
|
||||
("1080p", "3:4"): (1248, 1664),
|
||||
("1080p", "9:16"): (1080, 1920),
|
||||
("1080p", "21:9"): (2206, 946),
|
||||
}
|
||||
|
||||
#: 2.5 与 2.0 系列**唯一**不同的两格:480p 的 16:9 与 9:16。
|
||||
_OVERRIDES_2_5: dict[tuple[str, str], tuple[int, int]] = {
|
||||
("480p", "16:9"): (854, 480),
|
||||
("480p", "9:16"): (480, 854),
|
||||
}
|
||||
|
||||
_OVERRIDES_2_0: dict[tuple[str, str], tuple[int, int]] = {
|
||||
("480p", "16:9"): (864, 496),
|
||||
("480p", "9:16"): (496, 864),
|
||||
# 4k 仅 2.0 系列支持。
|
||||
("4k", "16:9"): (3840, 2160),
|
||||
("4k", "4:3"): (3326, 2494),
|
||||
("4k", "1:1"): (2880, 2880),
|
||||
("4k", "3:4"): (2494, 3326),
|
||||
("4k", "9:16"): (2160, 3840),
|
||||
("4k", "21:9"): (4398, 1886),
|
||||
}
|
||||
|
||||
_BY_FAMILY: dict[int, dict[tuple[str, str], tuple[int, int]]] = {
|
||||
id(SEEDANCE_2_5): {**_SHARED, **_OVERRIDES_2_5},
|
||||
id(SEEDANCE_2_0): {**_SHARED, **_OVERRIDES_2_0},
|
||||
}
|
||||
|
||||
|
||||
def pixels_for(
|
||||
limits: ModelLimits | None, resolution: str | None, ratio: str | None
|
||||
) -> tuple[int | None, int | None]:
|
||||
"""查像素宽高;查不到返回 `(None, None)`。
|
||||
|
||||
查不到的三种情形都**返回 None 而不猜**——一个错的尺寸比留空更糟,它会让
|
||||
客户端按错误的比例渲染,而且看不出是错的:
|
||||
|
||||
1. **`ratio` 为 `adaptive`**:厂商在图生视频时自动适配首帧图片的比例。查询
|
||||
响应是否把它解析成具体比例、抑或原样返回 `adaptive`,文档未明确说明;
|
||||
原样返回时本表无从查起。
|
||||
2. **未知模型族**:能力表放行了未知模型(见 `capabilities.py`),但像素值
|
||||
无法凭空推断。
|
||||
3. **该模型不支持的组合**:如 2.5 的 4k——本不该发生(提交前已预校验),
|
||||
出现即说明厂商行为与文档不符,留空并让上层看到缺值。
|
||||
"""
|
||||
if limits is None or not resolution or not ratio:
|
||||
return (None, None)
|
||||
table = _BY_FAMILY.get(id(limits))
|
||||
if table is None:
|
||||
return (None, None)
|
||||
found = table.get((resolution, ratio))
|
||||
return found if found else (None, None)
|
||||
@@ -132,6 +132,9 @@ class SeedanceTask:
|
||||
id: str
|
||||
status: SeedanceStatus | None
|
||||
raw_status: str | None
|
||||
#: 任务实际使用的模型(形如 `doubao-seedance-2-0-260128`)。用于按模型族
|
||||
#: 换算像素宽高——同一适配器实例可能同时处理两个模型族的任务。
|
||||
model: str | None = None
|
||||
video_url: str | None = None
|
||||
last_frame_url: str | None = None
|
||||
duration_sec: int | None = None
|
||||
@@ -169,6 +172,7 @@ class SeedanceTask:
|
||||
id=str(payload.get("id") or ""),
|
||||
status=SeedanceStatus.parse(raw_status),
|
||||
raw_status=raw_status,
|
||||
model=payload.get("model"),
|
||||
video_url=content.get("video_url"),
|
||||
last_frame_url=content.get("last_frame_url"),
|
||||
duration_sec=payload.get("duration"),
|
||||
|
||||
@@ -24,7 +24,10 @@ from app.domain.generation import (
|
||||
)
|
||||
from app.providers.base import GenerationProvider
|
||||
|
||||
from .capabilities import limits_for
|
||||
from .capabilities import validate as validate_limits
|
||||
from .client import ArkClient
|
||||
from .dimensions import pixels_for
|
||||
from .models import (
|
||||
ASSET_URL_TTL_SEC,
|
||||
TASKS_PATH,
|
||||
@@ -88,6 +91,17 @@ class SeedanceVideoProvider(GenerationProvider):
|
||||
model = request.model or self._default_model
|
||||
if not model:
|
||||
raise ValueError("未指定模型:请求未携带 model 且 ARK_VIDEO_MODEL 未配置")
|
||||
|
||||
# 本地预校验:已接入的两个模型限制不同(2.5 时长到 30 秒但无 4k,2.0 有 4k
|
||||
# 但时长上限 15 秒;2.5 图生视频时 ratio 锁 adaptive)。超范围的请求会被厂商
|
||||
# 400 拒绝——那要花一次往返、计入 RPM,并表现为一个失败的任务。未知模型放行。
|
||||
validate_limits(
|
||||
model,
|
||||
resolution=request.resolution,
|
||||
duration_sec=request.duration_sec,
|
||||
ratio=request.ratio,
|
||||
has_first_frame=bool(request.input_url),
|
||||
)
|
||||
return SeedanceSubmission(
|
||||
model=model,
|
||||
prompt=request.prompt,
|
||||
@@ -137,6 +151,14 @@ class SeedanceVideoProvider(GenerationProvider):
|
||||
),
|
||||
)
|
||||
|
||||
# 像素宽高由 (模型族, resolution, ratio) 查静态对照表得到——厂商只返回后两者,
|
||||
# 不返回像素值。查表不需要下载产物也不需要解码,故 widthPx/heightPx 在 AI
|
||||
# 视频产物上可零成本填真值(ADR-023 §11)。查不到时留空而不猜:一个错的
|
||||
# 尺寸会让客户端按错误比例渲染,且看不出是错的。
|
||||
width_px, height_px = pixels_for(
|
||||
limits_for(self._model_of(task)), task.resolution, task.ratio
|
||||
)
|
||||
|
||||
return GenerationResult(
|
||||
provider_request_id=request_id,
|
||||
state=state,
|
||||
@@ -148,14 +170,23 @@ class SeedanceVideoProvider(GenerationProvider):
|
||||
cover_url=task.last_frame_url,
|
||||
ratio=task.ratio,
|
||||
resolution=task.resolution,
|
||||
# 像素尺寸留空:火山只返回 ratio 与 resolution,换算需要一张随模型族
|
||||
# 变化的对照表。在 V7 目录种子定下模型族之前不编造尺寸。
|
||||
width_px=width_px,
|
||||
height_px=height_px,
|
||||
duration_ms=task.duration_ms,
|
||||
output_format=task.output_format,
|
||||
failure=failure,
|
||||
billing_tokens=task.completion_tokens,
|
||||
)
|
||||
|
||||
def _model_of(self, task: SeedanceTask) -> str:
|
||||
"""任务所用的模型。
|
||||
|
||||
查询响应带 `model` 字段(形如 `doubao-seedance-2-0-260128`),优先用它——
|
||||
这样同一个适配器实例处理不同模型的任务时,像素换算也能各按其族。缺失时
|
||||
才回落到配置的默认模型。
|
||||
"""
|
||||
return task.model or self._default_model
|
||||
|
||||
@staticmethod
|
||||
def _map_state(task: SeedanceTask) -> TaskState:
|
||||
match task.status:
|
||||
|
||||
@@ -0,0 +1,412 @@
|
||||
openapi: 3.1.0
|
||||
|
||||
info:
|
||||
title: patbond-ai 内部接口
|
||||
version: 0.1.0
|
||||
description: |
|
||||
`patbond-creation`(Java)调用 `patbond-ai`(Python)执行 AI 生成的内部契约。
|
||||
|
||||
**这份契约不对客户端暴露**,与 `patbond-doc/docs/api/openapi.yaml` 是两份独立
|
||||
文档、独立版本号(ADR-023 §4)。刻意不合并:内部适配端点若进入公开契约,会污染
|
||||
其路径/操作计数,也会让「对客户端的承诺」与「对内部实现的约定」混为一谈。
|
||||
本文档也**不进** patbond-api 的四模块快照锁。
|
||||
|
||||
## 字段命名为 snake_case
|
||||
|
||||
与项目对客户端的 camelCase 契约不同。这是有意的:本服务是 Python,snake_case
|
||||
使 JSON 与领域 dataclass 字段逐一对应、无需转换层;Java 侧只需在这一个 Feign
|
||||
客户端上配置命名策略。**用 camelCase 反而要在两端都加转换。**
|
||||
|
||||
## 两类失败必须分开(本契约的核心设计)
|
||||
|
||||
| 类别 | 含义 | 形态 |
|
||||
| --- | --- | --- |
|
||||
| **service** | Java↔Python 之间的问题:鉴权失败、请求体非法、任务不存在、本服务内部错误、适配器未装配 | 非 2xx + `error.kind = "service"` |
|
||||
| **provider** | Python↔AI 服务商之间的问题:限流、配额耗尽、内容被拒、厂商内部错误 | 非 2xx + `error.kind = "provider"`,**并携带完整分类** |
|
||||
| **任务失败** | HTTP 调用本身成功,但生成任务最终失败 | **200** + `state = "failed"` + `failure` |
|
||||
|
||||
第三类刻意不走错误通道:查询一个失败的任务,**HTTP 请求是成功的**,失败的是
|
||||
任务。把它做成非 2xx 会让「查不到任务」与「任务失败了」难以区分。
|
||||
|
||||
判别字段是 `error.kind`,**不是 HTTP 状态码**。Java 侧据此分流:
|
||||
`service` 类属我方 bug 或配置问题(除 `internal_error`/`provider_unavailable`
|
||||
外均不可重试);`provider` 类一律以 `retry` 字段为准,不要自己解读厂商码。
|
||||
|
||||
## 一处与服务商形态有关的说明
|
||||
|
||||
**服务商可能是同步的。** 火山的视频接口本身异步(提交返回任务 ID);OpenAI 的
|
||||
图片接口是**同步**的——一次请求等到底,没有任务 ID 也没有查询接口。该差异由本
|
||||
服务在内部吸收(ADR-023 §4),故本契约对两者形状完全一致。
|
||||
|
||||
代价如实登记:同步型供应商的在途任务只能记在本服务内存里,**进程重启会丢失**,
|
||||
之后查询返回 `task_not_found`。那次调用可能已经完成并计费,而同步接口没有任务
|
||||
ID 可供事后查询,故该窗口**无法靠"再查一次"弥补**,只能由调用方的
|
||||
`max_attempts` 兜住。并发在途任务达上限时提交返回 `provider_unavailable`
|
||||
(503,可重试)而非 `invalid_request`。
|
||||
|
||||
## 鉴权
|
||||
|
||||
所有端点要求服务身份 JWT:`Authorization: Bearer <token>`。本服务强制校验
|
||||
`aud = patbond-ai` 与 `typ = service`(ADR-023 §5)。用户 token 一律拒绝——
|
||||
该密钥对与用户 token 的密钥对是独立的两套。
|
||||
|
||||
servers:
|
||||
- url: http://127.0.0.1:8000
|
||||
description: 本机开发
|
||||
- url: http://patbond-ai:8000
|
||||
description: 容器网络内
|
||||
|
||||
security:
|
||||
- serviceJwt: []
|
||||
|
||||
paths:
|
||||
/internal/generation/tasks:
|
||||
post:
|
||||
summary: 提交生成任务
|
||||
operationId: submitGenerationTask
|
||||
description: |
|
||||
把任务交给上游服务商,返回**服务商的任务 ID**。
|
||||
|
||||
**不阻塞等待生成完成。** 但会阻塞一次到服务商的短往返——服务商的提交接口
|
||||
本身是异步的(只入队即返回任务 ID),所以这一跳很快。
|
||||
|
||||
调用方**必须在上游产生任何计费动作之前**把返回的 `provider_request_id`
|
||||
落库:火山不支持幂等键,这是提交超时后唯一能判断「是否已提交」的依据。
|
||||
|
||||
**一个无法消除的窗口**:若本服务到服务商的调用超时,调用方会收到
|
||||
`provider` 类错误且拿不到 ID,而服务商**可能已经创建了任务**。无幂等键可用,
|
||||
该窗口只能由 `max_attempts` 兜住(ADR-023 §4 已如实登记)。服务商支持
|
||||
回调,将来启用回调可作为孤儿任务的回收通道。
|
||||
requestBody:
|
||||
required: true
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/SubmitRequest'
|
||||
responses:
|
||||
'200':
|
||||
description: 已提交
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/SubmitResponse'
|
||||
'400':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
'401':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
'502':
|
||||
$ref: '#/components/responses/ProviderError'
|
||||
'503':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
|
||||
/internal/generation/tasks/{provider_request_id}:
|
||||
get:
|
||||
summary: 查询生成任务
|
||||
operationId: getGenerationTask
|
||||
description: |
|
||||
路径参数是**服务商的任务 ID**,不是我方 `job_id`。
|
||||
|
||||
这是刻意的:本服务不持有任何状态,唯一能用来问服务商的就是这个 ID。若用
|
||||
`job_id`,本服务就得存一份 `job_id → 厂商 ID` 的映射,「无状态」不再成立,
|
||||
且会多出一个「Java 有 ID 但查不到任何东西」的中间态。
|
||||
|
||||
注意火山**只保留最近 7 天**的任务记录,更早的任务会以 `provider` 类错误
|
||||
返回。
|
||||
parameters:
|
||||
- name: provider_request_id
|
||||
in: path
|
||||
required: true
|
||||
schema:
|
||||
type: string
|
||||
example: cgt-20260920-abc123
|
||||
responses:
|
||||
'200':
|
||||
description: |
|
||||
任务当前状态。**任务失败也是 200** —— HTTP 调用成功了,失败的是任务。
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/TaskResponse'
|
||||
'401':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
'404':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
'502':
|
||||
$ref: '#/components/responses/ProviderError'
|
||||
'503':
|
||||
$ref: '#/components/responses/ServiceError'
|
||||
|
||||
components:
|
||||
securitySchemes:
|
||||
serviceJwt:
|
||||
type: http
|
||||
scheme: bearer
|
||||
bearerFormat: JWT
|
||||
description: |
|
||||
由 `patbond-creation` 用**独立于用户 token 的**密钥对签发。claim 约定:
|
||||
`iss=patbond-api`、`aud=patbond-ai`、`sub=service:creation-worker`、
|
||||
`typ=service`、短 `exp`(5 分钟)、带 `jti`。
|
||||
|
||||
responses:
|
||||
ServiceError:
|
||||
description: Java↔Python 之间的问题
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ServiceErrorEnvelope'
|
||||
ProviderError:
|
||||
description: Python↔服务商之间的问题,携带完整分类
|
||||
content:
|
||||
application/json:
|
||||
schema:
|
||||
$ref: '#/components/schemas/ProviderErrorEnvelope'
|
||||
|
||||
schemas:
|
||||
SubmitRequest:
|
||||
type: object
|
||||
required: [job_id, media_kind, model]
|
||||
description: |
|
||||
**刻意不含 `user_id`**(数据最小化,ADR-023 §5):本服务干活只需要任务参数。
|
||||
需要给服务商做滥用检测时传的是 `safety_identifier`——一个哈希,不是明文 ID。
|
||||
properties:
|
||||
job_id:
|
||||
type: string
|
||||
format: uuid
|
||||
description: |
|
||||
我方任务 ID。**本服务不用它查任何东西**,仅用于日志关联,使一条报错能
|
||||
对应回 `creation.generation_jobs` 的行。
|
||||
media_kind:
|
||||
type: string
|
||||
enum: [image, video]
|
||||
description: 决定路由到哪个适配器。
|
||||
model:
|
||||
type: string
|
||||
description: |
|
||||
服务商侧的模型标识,取自任务行冻结的 `provider_model_snapshot`。
|
||||
**本服务不自行选模型**:选型是目录的职责,且任务行里的快照必须与实际
|
||||
调用的模型一致,否则事后无法对账。
|
||||
example: doubao-seedance-2-5-260628
|
||||
prompt:
|
||||
type: [string, 'null']
|
||||
maxLength: 10000
|
||||
description: 与 `input_url` **至少要有一个**,都为空则本地即拒。
|
||||
input_url:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
参考图/首帧的**公网可达** URL。给了即图生视频,不给即文生视频
|
||||
(参考图选填,决策 A2)。
|
||||
ratio:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
缺省 `adaptive`。**会按模型限制本地预校验**:Seedance 2.5 在图生视频时
|
||||
只支持 `adaptive`,指定具体比例会被本服务直接拒(省掉一次往返)。
|
||||
example: adaptive
|
||||
resolution:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
缺省 `720p`。按模型预校验:2.5 无 4k,2.0 系列有。**视频路径专用**。
|
||||
example: 720p
|
||||
width_px:
|
||||
type: [integer, 'null']
|
||||
description: |
|
||||
**图片路径专用**的像素宽。与 `height_px` 必须同时给,否则交由模型自选
|
||||
(半个尺寸不如不给)。
|
||||
|
||||
两家服务商的尺寸表达方式不同:火山用「分辨率档 + 宽高比」
|
||||
(`resolution` + `ratio`),OpenAI 用 `WIDTHxHEIGHT` 像素串。契约同时
|
||||
保留两种而**不强行统一**——任何统一都会在某一侧失真:把 720p+16:9 折成
|
||||
像素会丢掉「档位」语义,把像素折成比例会丢掉精确值。
|
||||
|
||||
OpenAI 侧的算术约束会在本服务**本地预校验**:宽高都须被 16 整除、
|
||||
宽高比在 1:3 ~ 3:1 之间、不超过 3840x2160(超过 2560x1440 属服务商
|
||||
标注的实验性范围)。违规请求以 `invalid_request` 拒绝,不花一次往返。
|
||||
height_px:
|
||||
type: [integer, 'null']
|
||||
description: 见 `width_px`。
|
||||
duration_sec:
|
||||
type: [integer, 'null']
|
||||
description: |
|
||||
缺省 5。按模型预校验:2.5 为 `[4,30]`,2.0 系列为 `[4,15]`;
|
||||
`-1` 表示由模型自选,不参与范围校验。
|
||||
generate_audio:
|
||||
type: boolean
|
||||
default: true
|
||||
watermark:
|
||||
type: boolean
|
||||
default: false
|
||||
safety_identifier:
|
||||
type: [string, 'null']
|
||||
maxLength: 64
|
||||
description: |
|
||||
终端用户标识,供服务商做滥用检测。**必须是哈希值,不可传明文用户 ID**。
|
||||
callback_url:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
预留,当前不启用(ADR-023 §4:轮询为保证,回调为优化)。留此字段使
|
||||
将来启用无需改契约。
|
||||
upload_target:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
产物的预签名 PUT 地址。**当前不启用**——转存阻塞于对象存储决策
|
||||
(ADR-023 §7)。启用后本服务会把产物直传该地址,从而无需持有任何
|
||||
对象存储凭证。
|
||||
|
||||
SubmitResponse:
|
||||
type: object
|
||||
required: [provider_request_id]
|
||||
properties:
|
||||
provider_request_id:
|
||||
type: string
|
||||
description: 服务商任务 ID。调用方须在上游计费前落库。
|
||||
example: cgt-20260920-abc123
|
||||
|
||||
TaskResponse:
|
||||
type: object
|
||||
required: [provider_request_id, state, progress]
|
||||
properties:
|
||||
provider_request_id:
|
||||
type: string
|
||||
state:
|
||||
type: string
|
||||
enum: [pending, running, succeeded, failed, cancelled]
|
||||
description: |
|
||||
**刻意比服务商的状态集更小。** 火山有 queued/running/cancelled/
|
||||
succeeded/failed/expired 六个:`queued` 映射为 `pending`,`expired`
|
||||
归入 `failed`(它是超时,属失败的一种,重试策略由 `failure` 表达,
|
||||
不需要独立状态)。
|
||||
progress:
|
||||
type: integer
|
||||
minimum: 0
|
||||
maximum: 100
|
||||
description: |
|
||||
火山不返回百分比,故成功前恒 0、成功即 100。**这不是缺陷**:客户端
|
||||
契约本就不承诺平滑进度(ADR-023 §10),一个诚实的 0 不该被读成「卡住」。
|
||||
asset_url:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
产物 URL。**可能是有时效的服务商地址**——火山为 24 小时。调用方必须在
|
||||
时效内转存,**不可长期保存此值**(存进 `media.assets.external_url`
|
||||
会在 24 小时后变死链)。
|
||||
asset_url_expires_in_sec:
|
||||
type: [integer, 'null']
|
||||
description: |
|
||||
`asset_url` 的剩余有效期。与 URL 一起给出,使调用方没有理由把它当
|
||||
永久地址。火山为 86400。
|
||||
cover_url:
|
||||
type: [string, 'null']
|
||||
description: |
|
||||
封面图 URL。火山的尾帧图(png、尺寸与视频一致、无水印)可免费充当此
|
||||
用途,省掉抽帧。
|
||||
ratio:
|
||||
type: [string, 'null']
|
||||
resolution:
|
||||
type: [string, 'null']
|
||||
width_px:
|
||||
type: [integer, 'null']
|
||||
description: |
|
||||
由 (模型族, resolution, ratio) 查静态对照表得到,**不需要下载产物或
|
||||
解码**。`ratio` 为 `adaptive` 或模型族未知时为 `null`——留空而不猜,
|
||||
一个错的尺寸会让客户端按错误比例渲染且看不出是错的。
|
||||
height_px:
|
||||
type: [integer, 'null']
|
||||
duration_ms:
|
||||
type: [integer, 'null']
|
||||
description: |
|
||||
有 `frames`+`fps` 时按帧精确换算;只有服务商的整数秒时退回它,而那是
|
||||
「帧数/24 向下取整」的**约数**(最多可能短 1 秒)。两者都缺时为 null,
|
||||
不编 0。
|
||||
output_format:
|
||||
type: [string, 'null']
|
||||
example: mp4
|
||||
billing_tokens:
|
||||
type: [integer, 'null']
|
||||
description: 计费对账依据(火山为 `usage.completion_tokens`)。
|
||||
failure:
|
||||
oneOf:
|
||||
- $ref: '#/components/schemas/Failure'
|
||||
- type: 'null'
|
||||
description: |
|
||||
仅 `state = failed` 时非空。**`failed` 必然携带它**——否则调用方无法
|
||||
决定重试策略,只能盲目重试或盲目放弃,两者都错。
|
||||
|
||||
Failure:
|
||||
type: object
|
||||
required: [retry, user_facing, provider_code]
|
||||
properties:
|
||||
retry:
|
||||
type: string
|
||||
enum: [never, backoff, quota]
|
||||
description: |
|
||||
**Worker 据此决定重试,不必解读服务商码。**
|
||||
`never`:同样输入必然同样结果(参数错、内容被拒、鉴权失败)。
|
||||
`backoff`:限流、突发流量保护、服务过载,退避几秒到几十秒。
|
||||
`quota`:额度耗尽,要等到下一个计费周期或充值——不该把重试预算花在
|
||||
这上面,应直接终态失败并让用户看到出口。
|
||||
user_facing:
|
||||
type: string
|
||||
enum:
|
||||
[quota_exhausted, rate_limited, content_rejected, server_error, upstream_unavailable]
|
||||
description: |
|
||||
映射到客户端呈现。`quota_exhausted` 与 `rate_limited` **必须分开**:
|
||||
前者等的是明天或充值(弹 sheet 给出口),后者等的是几秒(弹 SnackBar
|
||||
可自动重试)。合成一类必然有一半场景的文案和出口是错的(ADR-023 §9)。
|
||||
provider_code:
|
||||
type: string
|
||||
description: |
|
||||
原始服务商错误码,**原样保留、绝不丢弃**——排查时往往是唯一线索。
|
||||
上层不应据此分支判断(那是适配层的职责),只记录与展示给开发者。
|
||||
example: OutputVideoSensitiveContentDetected
|
||||
message:
|
||||
type: string
|
||||
description: 服务商原始错误描述,仅供排查与开发者可见的日志。
|
||||
|
||||
ServiceErrorEnvelope:
|
||||
type: object
|
||||
required: [error]
|
||||
properties:
|
||||
error:
|
||||
type: object
|
||||
required: [kind, code, message]
|
||||
properties:
|
||||
kind:
|
||||
type: string
|
||||
const: service
|
||||
code:
|
||||
type: string
|
||||
enum:
|
||||
- unauthenticated # 服务票缺失/过期/aud 或 typ 不符
|
||||
- invalid_request # 请求体不合契约,或参数超出模型限制
|
||||
- task_not_found # 路径里的服务商任务 ID 不存在
|
||||
- provider_unavailable # 适配器未装配(如缺凭证)或 provider_code 未知
|
||||
- internal_error # 本服务 bug
|
||||
message:
|
||||
type: string
|
||||
|
||||
ProviderErrorEnvelope:
|
||||
type: object
|
||||
required: [error]
|
||||
description: |
|
||||
服务商调用失败。**与任务失败(200 + state=failed)是两回事**:这里是调用
|
||||
没成功,那里是调用成功但任务最终失败。
|
||||
properties:
|
||||
error:
|
||||
type: object
|
||||
required: [kind, retry, user_facing, provider_code, message]
|
||||
description: |
|
||||
字段与 `Failure` 一致,故 Java 侧可复用同一段分类处置逻辑,无需为
|
||||
「提交时被限流」和「任务失败于限流」写两套分支。
|
||||
properties:
|
||||
kind:
|
||||
type: string
|
||||
const: provider
|
||||
retry:
|
||||
type: string
|
||||
enum: [never, backoff, quota]
|
||||
user_facing:
|
||||
type: string
|
||||
enum:
|
||||
[quota_exhausted, rate_limited, content_rejected, server_error, upstream_unavailable]
|
||||
provider_code:
|
||||
type: string
|
||||
message:
|
||||
type: string
|
||||
@@ -0,0 +1,27 @@
|
||||
[project]
|
||||
name = "patbond-ai"
|
||||
version = "0.1.0"
|
||||
description = "Patbond AI 生成服务:调用 AI 服务商 API 完成图片与视频生成"
|
||||
requires-python = ">=3.11"
|
||||
|
||||
[tool.ruff]
|
||||
# 与 CI 门禁一致(.gitea/workflows/ci.yml)。
|
||||
line-length = 88
|
||||
|
||||
[tool.ruff.lint.flake8-bugbear]
|
||||
# `Depends()` / `Query()` 等放在参数默认值里**正是 FastAPI 的标准写法**,
|
||||
# B008 在此是误报。为迎合规则把依赖声明搬进函数体会失去类型提示与自动文档,
|
||||
# 是本末倒置——故按 ruff 提供的正规方式声明这些调用为可安全求值,
|
||||
# 而不是在每个端点上撒 `# noqa`(后者迟早被用来压制真正的问题)。
|
||||
extend-immutable-calls = [
|
||||
"fastapi.Depends",
|
||||
"fastapi.Query",
|
||||
"fastapi.Path",
|
||||
"fastapi.Header",
|
||||
"fastapi.Body",
|
||||
]
|
||||
|
||||
[tool.pytest.ini_options]
|
||||
# 让 `pytest` 从仓库根目录直接跑通,无需先安装包。
|
||||
pythonpath = ["."]
|
||||
testpaths = ["tests"]
|
||||
@@ -2,3 +2,4 @@
|
||||
# 与 CI 门禁同一套工具(.gitea/workflows/ci.yml)
|
||||
ruff>=0.6
|
||||
pytest>=8.0
|
||||
pyyaml>=6.0
|
||||
|
||||
@@ -1,3 +1,8 @@
|
||||
# 运行时依赖。版本下限按实测可用版本给,不锁死上限——本服务只用 httpx 的
|
||||
# 基础同步 API,其小版本变更风险低。
|
||||
httpx>=0.27
|
||||
# HTTP 服务(内部接口)。uvicorn 是运行时服务器,非库依赖。
|
||||
fastapi>=0.115
|
||||
uvicorn>=0.30
|
||||
# 服务身份 JWT 的 RS256 校验;[crypto] 带入 cryptography。
|
||||
pyjwt[crypto]>=2.9
|
||||
|
||||
@@ -0,0 +1,177 @@
|
||||
"""手写契约 `openapi-internal.yaml` 与实际实现的一致性守卫。
|
||||
|
||||
**为什么需要它**:契约是手写的,实现是另一份代码。本项目开工审计已经证明「没有
|
||||
门禁的契约必然漂移」——四模块快照锁实际不存在,结果白名单 42/41 之误四份文档一路
|
||||
不可见,整整一个迭代没人发现。内部契约同样是契约,同样需要门禁。
|
||||
|
||||
**不做的事**:不比对 FastAPI 自动生成的 `/openapi.json` 与手写 yaml 的字节一致性。
|
||||
两者永远不会字节相同(描述、示例、`$ref` 组织方式都不同),逐字比对只会制造一个
|
||||
天天红的门禁,然后被人加 skip 绕过。这里比对的是**有实质意义的东西**:路径、方法、
|
||||
字段集、枚举取值。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pathlib
|
||||
|
||||
import yaml
|
||||
|
||||
from app.api.errors import ServiceErrorCode
|
||||
from app.api.main import create_app
|
||||
from app.api.schemas import FailureModel, SubmitRequest, SubmitResponse, TaskResponse
|
||||
from app.config import ServiceAuthSettings, Settings, VolcengineSettings
|
||||
from app.domain.generation import Retry, TaskState, UserFacing
|
||||
|
||||
CONTRACT = yaml.safe_load(
|
||||
(
|
||||
pathlib.Path(__file__).resolve().parent.parent.parent / "openapi-internal.yaml"
|
||||
).read_text(encoding="utf-8")
|
||||
)
|
||||
|
||||
|
||||
def schema_properties(name: str) -> set[str]:
|
||||
return set(CONTRACT["components"]["schemas"][name]["properties"])
|
||||
|
||||
|
||||
def app_routes() -> set[tuple[str, str]]:
|
||||
"""实际对外暴露的 `/internal` 端点集合。
|
||||
|
||||
取自 `app.openapi()` 而非遍历 `app.routes`:后者在不同 FastAPI 版本里结构不同
|
||||
(本版本的 `include_router` 会包一层内部 wrapper,路由并不平铺在 `app.routes`
|
||||
上),依赖它会让这个守卫在升级框架时莫名失效——而一个悄悄失效的门禁比没有
|
||||
门禁更糟。`openapi()` 是「究竟暴露了什么」的权威视图。
|
||||
"""
|
||||
app = create_app(
|
||||
Settings(
|
||||
volcengine=VolcengineSettings(api_key="dummy-api-key-for-tests"),
|
||||
service_auth=ServiceAuthSettings(public_key=""),
|
||||
)
|
||||
)
|
||||
return {
|
||||
(method.upper(), path)
|
||||
for path, operations in app.openapi()["paths"].items()
|
||||
if path.startswith("/internal")
|
||||
for method in operations
|
||||
}
|
||||
|
||||
|
||||
class TestPathsMatch:
|
||||
def test_every_contract_path_is_implemented(self):
|
||||
implemented = app_routes()
|
||||
for path, operations in CONTRACT["paths"].items():
|
||||
for method in operations:
|
||||
assert (method.upper(), path) in implemented, (
|
||||
f"契约声明了 {method.upper()} {path} 但实现里没有"
|
||||
)
|
||||
|
||||
def test_no_undeclared_internal_route_exists(self):
|
||||
"""实现里不得有契约之外的 /internal 端点。
|
||||
|
||||
未声明的内部端点是最容易被遗忘的表面——它不在契约里,评审看不到,
|
||||
但它确实对外开放。
|
||||
"""
|
||||
declared = {
|
||||
(method.upper(), path)
|
||||
for path, operations in CONTRACT["paths"].items()
|
||||
for method in operations
|
||||
}
|
||||
assert app_routes() <= declared
|
||||
|
||||
|
||||
class TestFieldSetsMatch:
|
||||
"""字段集漂移是最隐蔽的一种:多一个字段无人察觉,少一个字段调用方拿到 None。"""
|
||||
|
||||
def test_submit_request(self):
|
||||
assert schema_properties("SubmitRequest") == set(SubmitRequest.model_fields)
|
||||
|
||||
def test_submit_response(self):
|
||||
assert schema_properties("SubmitResponse") == set(SubmitResponse.model_fields)
|
||||
|
||||
def test_task_response(self):
|
||||
assert schema_properties("TaskResponse") == set(TaskResponse.model_fields)
|
||||
|
||||
def test_failure(self):
|
||||
assert schema_properties("Failure") == set(FailureModel.model_fields)
|
||||
|
||||
def test_all_field_names_are_snake_case(self):
|
||||
"""契约定为 snake_case(与对客户端的 camelCase 契约不同,见契约说明)。
|
||||
|
||||
混入一个 camelCase 字段就会让「JSON 与 dataclass 字段逐一对应、无需转换层」
|
||||
这个前提失效。
|
||||
"""
|
||||
for name in ("SubmitRequest", "SubmitResponse", "TaskResponse", "Failure"):
|
||||
for field in schema_properties(name):
|
||||
assert field.islower() or "_" in field, (
|
||||
f"{name}.{field} 不是 snake_case"
|
||||
)
|
||||
assert not any(char.isupper() for char in field), (
|
||||
f"{name}.{field} 含大写字母"
|
||||
)
|
||||
|
||||
|
||||
class TestEnumsMatch:
|
||||
"""枚举漂移的后果是调用方收到一个它没处理的取值——而那通常走到默认分支。"""
|
||||
|
||||
def test_task_states(self):
|
||||
declared = set(
|
||||
CONTRACT["components"]["schemas"]["TaskResponse"]["properties"]["state"][
|
||||
"enum"
|
||||
]
|
||||
)
|
||||
assert declared == {state.value for state in TaskState}
|
||||
|
||||
def test_retry_strategies(self):
|
||||
declared = set(
|
||||
CONTRACT["components"]["schemas"]["Failure"]["properties"]["retry"]["enum"]
|
||||
)
|
||||
assert declared == {retry.value for retry in Retry}
|
||||
|
||||
def test_user_facing_categories(self):
|
||||
declared = set(
|
||||
CONTRACT["components"]["schemas"]["Failure"]["properties"]["user_facing"][
|
||||
"enum"
|
||||
]
|
||||
)
|
||||
assert declared == {category.value for category in UserFacing}
|
||||
|
||||
def test_service_error_codes(self):
|
||||
declared = set(
|
||||
CONTRACT["components"]["schemas"]["ServiceErrorEnvelope"]["properties"][
|
||||
"error"
|
||||
]["properties"]["code"]["enum"]
|
||||
)
|
||||
assert declared == {code.code for code in ServiceErrorCode}
|
||||
|
||||
def test_provider_error_reuses_the_failure_classification_fields(self):
|
||||
"""`provider` 错误信封与 `Failure` 的分类字段必须一致。
|
||||
|
||||
这是刻意设计:调用方因此可以复用同一段处置逻辑,不必为「提交时被限流」
|
||||
与「任务失败于限流」写两套分支。两边若漂移,那份复用就会悄悄失效。
|
||||
"""
|
||||
envelope = set(
|
||||
CONTRACT["components"]["schemas"]["ProviderErrorEnvelope"]["properties"][
|
||||
"error"
|
||||
]["properties"]
|
||||
)
|
||||
failure = schema_properties("Failure")
|
||||
assert envelope == failure | {"kind"}
|
||||
|
||||
|
||||
class TestRequiredFieldsMatch:
|
||||
def test_submit_request_required(self):
|
||||
declared = set(CONTRACT["components"]["schemas"]["SubmitRequest"]["required"])
|
||||
actual = {
|
||||
name
|
||||
for name, field in SubmitRequest.model_fields.items()
|
||||
if field.is_required()
|
||||
}
|
||||
assert declared == actual
|
||||
|
||||
def test_task_response_required(self):
|
||||
declared = set(CONTRACT["components"]["schemas"]["TaskResponse"]["required"])
|
||||
actual = {
|
||||
name
|
||||
for name, field in TaskResponse.model_fields.items()
|
||||
if field.is_required()
|
||||
}
|
||||
assert declared == actual
|
||||
@@ -0,0 +1,512 @@
|
||||
"""内部接口的测试:鉴权边界、两类错误信封、端到端编排。
|
||||
|
||||
全部离线——服务商调用被 `httpx.MockTransport` 拦下,服务票用测试期临时生成的
|
||||
RSA 密钥对签发。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
import jwt
|
||||
import pytest
|
||||
from cryptography.hazmat.primitives import serialization
|
||||
from cryptography.hazmat.primitives.asymmetric import rsa
|
||||
from fastapi.testclient import TestClient
|
||||
|
||||
from app.api.main import create_app
|
||||
from app.config import ServiceAuthSettings, Settings, VolcengineSettings
|
||||
from app.providers.registry import ProviderRegistry
|
||||
from app.providers.volcengine.client import ArkClient
|
||||
from app.providers.volcengine.video import SeedanceVideoProvider
|
||||
|
||||
FAKE_API_KEY = "dummy-api-key-for-tests"
|
||||
MODEL_2_5 = "doubao-seedance-2-5-260628"
|
||||
MODEL_2_0 = "doubao-seedance-2-0-260128"
|
||||
|
||||
|
||||
# ----------------------------------------------------------------- 夹具
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
def keypair() -> tuple[str, str]:
|
||||
"""测试期临时 RSA 密钥对(私钥签票、公钥配给服务)。
|
||||
|
||||
现场生成而非硬编码 PEM:密钥形态的 PEM 写进仓库会被防泄漏扫描拦下,而且
|
||||
那条规则是对的。
|
||||
"""
|
||||
key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
|
||||
private_pem = key.private_bytes(
|
||||
serialization.Encoding.PEM,
|
||||
serialization.PrivateFormat.PKCS8,
|
||||
serialization.NoEncryption(),
|
||||
).decode()
|
||||
public_pem = (
|
||||
key.public_key()
|
||||
.public_bytes(
|
||||
serialization.Encoding.PEM,
|
||||
serialization.PublicFormat.SubjectPublicKeyInfo,
|
||||
)
|
||||
.decode()
|
||||
)
|
||||
return private_pem, public_pem
|
||||
|
||||
|
||||
def service_token(private_pem: str, **overrides) -> str:
|
||||
"""签一张服务票;`overrides` 用于构造各种不合格的票。"""
|
||||
import time
|
||||
|
||||
claims = {
|
||||
"iss": "patbond-api",
|
||||
"aud": "patbond-ai",
|
||||
"sub": "service:creation-worker",
|
||||
"typ": "service",
|
||||
"iat": int(time.time()),
|
||||
"exp": int(time.time()) + 300,
|
||||
"jti": "test-jti",
|
||||
}
|
||||
claims.update(overrides)
|
||||
return jwt.encode(claims, private_pem, algorithm="RS256")
|
||||
|
||||
|
||||
def client_for(keypair, handler, *, with_provider: bool = True) -> TestClient:
|
||||
"""建一个测试客户端,服务商调用被 handler 拦下。"""
|
||||
_, public_pem = keypair
|
||||
settings = Settings(
|
||||
volcengine=VolcengineSettings(api_key=FAKE_API_KEY if with_provider else ""),
|
||||
service_auth=ServiceAuthSettings(public_key=public_pem),
|
||||
)
|
||||
app = create_app(settings)
|
||||
# 换掉装配好的适配器,让它的 HTTP 走 MockTransport。
|
||||
if with_provider:
|
||||
app.state.registry = ProviderRegistry(
|
||||
[
|
||||
SeedanceVideoProvider(
|
||||
ArkClient(
|
||||
api_key=FAKE_API_KEY,
|
||||
base_url="https://ark.example.invalid/api/v3",
|
||||
client=httpx.Client(transport=httpx.MockTransport(handler)),
|
||||
)
|
||||
)
|
||||
]
|
||||
)
|
||||
else:
|
||||
app.state.registry = ProviderRegistry([])
|
||||
return TestClient(app, raise_server_exceptions=False)
|
||||
|
||||
|
||||
def auth(keypair, **overrides) -> dict[str, str]:
|
||||
private_pem, _ = keypair
|
||||
return {"Authorization": f"Bearer {service_token(private_pem, **overrides)}"}
|
||||
|
||||
|
||||
SUBMIT_BODY = {
|
||||
"job_id": "0199a1b2-c3d4-7000-8000-000000000001",
|
||||
"media_kind": "video",
|
||||
"model": MODEL_2_5,
|
||||
"prompt": "我的柯基在草地上奔跑",
|
||||
}
|
||||
|
||||
|
||||
# ------------------------------------------------------------- 鉴权边界
|
||||
|
||||
|
||||
class TestAuthentication:
|
||||
"""服务票是这一层的安全边界。失败一律返回同一个笼统原因,细节只进日志。"""
|
||||
|
||||
def test_missing_token_is_rejected(self, keypair):
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post("/internal/generation/tasks", json=SUBMIT_BODY)
|
||||
assert response.status_code == 401
|
||||
assert response.json()["error"]["kind"] == "service"
|
||||
assert response.json()["error"]["code"] == "unauthenticated"
|
||||
|
||||
def test_wrong_audience_is_rejected(self, keypair):
|
||||
"""`aud` 强制校验是独立密钥对方案唯一的意义所在。
|
||||
|
||||
Java 侧现有的 JwtVerifier 只验签名、不校验 aud——正因如此才必须用独立
|
||||
密钥对;如果本服务也只验签名,那对独立密钥就白搭了。
|
||||
"""
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json=SUBMIT_BODY,
|
||||
headers=auth(keypair, aud="someone-else"),
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_wrong_issuer_is_rejected(self, keypair):
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json=SUBMIT_BODY,
|
||||
headers=auth(keypair, iss="attacker"),
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_a_user_token_shape_is_rejected_even_if_signed_with_this_key(self, keypair):
|
||||
"""有人误把用户密钥配到这里时的第二道防线。
|
||||
|
||||
用户 token 没有 `typ=service`(Java 侧的 JwtSigner 也不签 aud),故即使
|
||||
签名能过也会被拦下。
|
||||
"""
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json=SUBMIT_BODY,
|
||||
headers=auth(keypair, typ=None),
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_expired_token_is_rejected(self, keypair):
|
||||
import time
|
||||
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json=SUBMIT_BODY,
|
||||
headers=auth(keypair, exp=int(time.time()) - 10),
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_missing_public_key_denies_everything_rather_than_allowing(self, keypair):
|
||||
"""配置缺失必须是「全拒」而非「全过」。
|
||||
|
||||
前者立刻暴露,后者是一扇敞开的门。与 Java 侧 InternalAuthFilter 未配置
|
||||
internal-token 时全拒 /internal/** 的既有选择一致。
|
||||
"""
|
||||
settings = Settings(
|
||||
volcengine=VolcengineSettings(api_key=FAKE_API_KEY),
|
||||
service_auth=ServiceAuthSettings(public_key=""),
|
||||
)
|
||||
client = TestClient(create_app(settings), raise_server_exceptions=False)
|
||||
response = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
)
|
||||
assert response.status_code == 401
|
||||
|
||||
def test_error_body_never_says_why(self, keypair):
|
||||
"""具体原因(签名错/过期/aud 不符)只进日志。
|
||||
|
||||
对方是我们自己的服务,真正原因在本服务日志里;把细节回给调用方只会在
|
||||
将来某天变成攻击者的提示。
|
||||
"""
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
body = client.post("/internal/generation/tasks", json=SUBMIT_BODY).json()
|
||||
message = body["error"]["message"]
|
||||
for leak in ("signature", "aud", "audience", "expired", "iss"):
|
||||
assert leak.lower() not in message.lower()
|
||||
|
||||
|
||||
class TestHealthNeedsNoToken:
|
||||
def test_health_is_outside_internal_and_unauthenticated(self, keypair):
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={}))
|
||||
response = client.get("/health")
|
||||
assert response.status_code == 200
|
||||
assert response.json()["status"] == "ok"
|
||||
|
||||
def test_health_reports_assembled_providers(self, keypair):
|
||||
"""缺凭证时适配器不装配,而那个症状(任务永远排队)离根因很远。
|
||||
|
||||
让探活直接说出来能省很多排查时间。
|
||||
"""
|
||||
with_provider = client_for(keypair, lambda r: httpx.Response(200, json={}))
|
||||
assert with_provider.get("/health").json()["providers"] == [
|
||||
"volcengine-seedance"
|
||||
]
|
||||
|
||||
without = client_for(
|
||||
keypair, lambda r: httpx.Response(200, json={}), with_provider=False
|
||||
)
|
||||
assert without.get("/health").json()["providers"] == []
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 提交
|
||||
|
||||
|
||||
class TestSubmit:
|
||||
def test_returns_provider_request_id(self, keypair):
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
assert request.url.path.endswith("/contents/generations/tasks")
|
||||
return httpx.Response(200, json={"id": "cgt-20260920-abc123"})
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
response = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
)
|
||||
assert response.status_code == 200
|
||||
assert response.json() == {"provider_request_id": "cgt-20260920-abc123"}
|
||||
|
||||
def test_field_names_are_snake_case(self, keypair):
|
||||
"""契约为 snake_case——camelCase 的请求体应被拒。
|
||||
|
||||
`extra="forbid"` 使拼错的字段立刻报错,而不是被静默忽略(后者表现为
|
||||
「我传了 durationSec 却没生效」这类极难查的问题)。
|
||||
"""
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json={**SUBMIT_BODY, "durationSec": 5},
|
||||
headers=auth(keypair),
|
||||
)
|
||||
assert response.status_code == 400
|
||||
assert response.json()["error"]["code"] == "invalid_request"
|
||||
|
||||
def test_user_id_is_not_accepted(self, keypair):
|
||||
"""契约刻意不含 user_id(数据最小化)。传了应当被拒而不是被忽略。"""
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={"id": "x"}))
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json={**SUBMIT_BODY, "user_id": "u-1"},
|
||||
headers=auth(keypair),
|
||||
)
|
||||
assert response.status_code == 400
|
||||
|
||||
def test_empty_content_is_rejected_locally(self, keypair):
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
body = {k: v for k, v in SUBMIT_BODY.items() if k != "prompt"}
|
||||
response = client.post(
|
||||
"/internal/generation/tasks", json=body, headers=auth(keypair)
|
||||
)
|
||||
assert response.status_code == 400
|
||||
assert "至少要有一个" in response.json()["error"]["message"]
|
||||
|
||||
def test_model_limit_violation_is_invalid_request_not_provider_error(self, keypair):
|
||||
"""超出模型限制的请求**根本没发出去**,故属 service/invalid_request。
|
||||
|
||||
归到 provider 类会让调用方以为是上游的问题而去退避重试——而重试同样的
|
||||
参数永远不会变好。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
response = client.post(
|
||||
"/internal/generation/tasks",
|
||||
json={**SUBMIT_BODY, "model": MODEL_2_0, "duration_sec": 20},
|
||||
headers=auth(keypair),
|
||||
)
|
||||
assert response.status_code == 400
|
||||
error = response.json()["error"]
|
||||
assert error["kind"] == "service"
|
||||
assert error["code"] == "invalid_request"
|
||||
assert "[4, 15]" in error["message"]
|
||||
|
||||
def test_no_adapter_yields_provider_unavailable(self, keypair):
|
||||
"""缺凭证 → 适配器未装配。503 而非 500:这是环境问题且可恢复。"""
|
||||
client = client_for(
|
||||
keypair, lambda r: httpx.Response(200, json={}), with_provider=False
|
||||
)
|
||||
response = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
)
|
||||
assert response.status_code == 503
|
||||
assert response.json()["error"]["code"] == "provider_unavailable"
|
||||
|
||||
|
||||
# ------------------------------------------------------- 两类错误的分界
|
||||
|
||||
|
||||
class TestTwoErrorKindsAreDistinct:
|
||||
"""这是契约的核心设计:调用方对两类失败的反应完全不同。"""
|
||||
|
||||
def test_vendor_rate_limit_is_a_provider_error_with_full_classification(
|
||||
self, keypair
|
||||
):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
429,
|
||||
json={
|
||||
"error": {
|
||||
"code": "RateLimitExceeded.EndpointRPMExceeded",
|
||||
"message": "RPM limit exceeded",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
response = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
)
|
||||
# 刻意不透传服务商的 429:429 既可能是「等几秒」也可能是「等到明天」,
|
||||
# 状态码表达不了这个区别,retry 才是权威信号。
|
||||
assert response.status_code == 502
|
||||
error = response.json()["error"]
|
||||
assert error["kind"] == "provider"
|
||||
assert error["retry"] == "backoff"
|
||||
assert error["user_facing"] == "rate_limited"
|
||||
assert error["provider_code"] == "RateLimitExceeded.EndpointRPMExceeded"
|
||||
|
||||
def test_quota_exhausted_carries_a_different_retry_than_rate_limit(self, keypair):
|
||||
"""同为服务商 429,但一个等几秒、一个等到明天。
|
||||
|
||||
调用方只看 retry 就能分开,不必解读厂商码。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
429,
|
||||
json={
|
||||
"error": {
|
||||
"code": "QuotaExceeded",
|
||||
"message": "Your account has exhausted its free trial quota.",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
error = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
).json()["error"]
|
||||
assert error["retry"] == "quota"
|
||||
assert error["user_facing"] == "quota_exhausted"
|
||||
|
||||
def test_kind_is_the_discriminator_not_the_status_code(self, keypair):
|
||||
"""两类错误的状态码会撞,`kind` 不会。
|
||||
|
||||
502 既可能是服务商挂了(provider),也可能是我们打错地址导致的解析失败
|
||||
(也归 provider,但 service 类错误同样可能出现 5xx)。判别必须看 kind。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(502, text="<html>bad gateway</html>")
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
body = client.post(
|
||||
"/internal/generation/tasks", json=SUBMIT_BODY, headers=auth(keypair)
|
||||
).json()
|
||||
assert body["error"]["kind"] == "provider"
|
||||
assert "kind" in body["error"]
|
||||
|
||||
|
||||
# ------------------------------------------------------------------ 查询
|
||||
|
||||
|
||||
#: 官方查询文档「返回格式」一节的原样示例(2.0 系列 720p + 16:9)。
|
||||
OFFICIAL_QUERY_PAYLOAD = {
|
||||
"id": "cgt-1",
|
||||
"model": MODEL_2_0,
|
||||
"status": "succeeded",
|
||||
"content": {
|
||||
"video_url": "https://v/out.mp4",
|
||||
"last_frame_url": "https://v/f.png",
|
||||
},
|
||||
"usage": {"completion_tokens": 108900, "total_tokens": 108900},
|
||||
"resolution": "720p",
|
||||
"ratio": "16:9",
|
||||
"duration": 5,
|
||||
"framespersecond": 24,
|
||||
}
|
||||
|
||||
|
||||
class TestGetTask:
|
||||
def test_succeeded_task_is_fully_serialised(self, keypair):
|
||||
client = client_for(
|
||||
keypair, lambda r: httpx.Response(200, json=OFFICIAL_QUERY_PAYLOAD)
|
||||
)
|
||||
body = client.get(
|
||||
"/internal/generation/tasks/cgt-1", headers=auth(keypair)
|
||||
).json()
|
||||
assert body["state"] == "succeeded"
|
||||
assert body["progress"] == 100
|
||||
assert body["asset_url"] == "https://v/out.mp4"
|
||||
assert body["cover_url"] == "https://v/f.png"
|
||||
assert (body["width_px"], body["height_px"]) == (1280, 720)
|
||||
assert body["duration_ms"] == 5000
|
||||
assert body["billing_tokens"] == 108900
|
||||
assert body["failure"] is None
|
||||
|
||||
def test_asset_url_always_carries_its_expiry(self, keypair):
|
||||
"""服务商 URL 只有 24 小时有效。时效随 URL 一起给出,调用方就没有理由
|
||||
把它当永久地址存进 external_url。"""
|
||||
client = client_for(
|
||||
keypair, lambda r: httpx.Response(200, json=OFFICIAL_QUERY_PAYLOAD)
|
||||
)
|
||||
body = client.get(
|
||||
"/internal/generation/tasks/cgt-1", headers=auth(keypair)
|
||||
).json()
|
||||
assert body["asset_url_expires_in_sec"] == 86400
|
||||
|
||||
def test_a_failed_task_is_200_not_an_error_response(self, keypair):
|
||||
"""查询一个失败的任务,**HTTP 调用是成功的**,失败的是任务。
|
||||
|
||||
做成非 2xx 会让「查不到任务」与「任务失败了」难以区分。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "cgt-1",
|
||||
"status": "failed",
|
||||
"content": {},
|
||||
"error": {
|
||||
"code": "OutputVideoSensitiveContentDetected",
|
||||
"message": "output may contain sensitive information",
|
||||
},
|
||||
},
|
||||
)
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
response = client.get("/internal/generation/tasks/cgt-1", headers=auth(keypair))
|
||||
assert response.status_code == 200
|
||||
body = response.json()
|
||||
assert body["state"] == "failed"
|
||||
assert body["failure"]["retry"] == "never"
|
||||
assert body["failure"]["user_facing"] == "content_rejected"
|
||||
assert body["failure"]["provider_code"] == "OutputVideoSensitiveContentDetected"
|
||||
|
||||
def test_vendor_404_becomes_task_not_found_so_the_caller_stops_polling(
|
||||
self, keypair
|
||||
):
|
||||
"""服务商只保留 7 天记录。查不到时调用方要做的是**停止轮询**,
|
||||
不是退避重试——故归 service/task_not_found 而非 provider。"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
404, json={"error": {"code": "NotFound.", "message": "not found"}}
|
||||
)
|
||||
|
||||
client = client_for(keypair, handler)
|
||||
response = client.get(
|
||||
"/internal/generation/tasks/cgt-gone", headers=auth(keypair)
|
||||
)
|
||||
assert response.status_code == 404
|
||||
error = response.json()["error"]
|
||||
assert error["kind"] == "service"
|
||||
assert error["code"] == "task_not_found"
|
||||
|
||||
def test_running_task_has_zero_progress(self, keypair):
|
||||
# 服务商不给百分比,故成功前恒 0。这不是缺陷——契约本就不承诺平滑进度。
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200, json={"id": "cgt-1", "status": "running", "content": {}}
|
||||
)
|
||||
|
||||
body = (
|
||||
client_for(keypair, handler)
|
||||
.get("/internal/generation/tasks/cgt-1", headers=auth(keypair))
|
||||
.json()
|
||||
)
|
||||
assert body["state"] == "running"
|
||||
assert body["progress"] == 0
|
||||
|
||||
def test_queued_maps_to_pending(self, keypair):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200, json={"id": "cgt-1", "status": "queued", "content": {}}
|
||||
)
|
||||
|
||||
body = (
|
||||
client_for(keypair, handler)
|
||||
.get("/internal/generation/tasks/cgt-1", headers=auth(keypair))
|
||||
.json()
|
||||
)
|
||||
assert body["state"] == "pending"
|
||||
|
||||
def test_get_requires_a_token_too(self, keypair):
|
||||
client = client_for(keypair, lambda r: httpx.Response(200, json={}))
|
||||
assert client.get("/internal/generation/tasks/cgt-1").status_code == 401
|
||||
@@ -0,0 +1,150 @@
|
||||
"""gpt-image-2 尺寸与参数约束的测试。
|
||||
|
||||
每个用例对应官方文档里的一条约束。这些约束若搞错,表现是用户提交后拿到一个失败
|
||||
任务(服务商 400),而本地测试全绿——所以值得逐条钉住。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from app.providers.openai.capabilities import (
|
||||
parse_size,
|
||||
validate_output_format,
|
||||
validate_quality,
|
||||
validate_size,
|
||||
)
|
||||
|
||||
|
||||
class TestStandardSizesAndAutoAlwaysPass:
|
||||
"""文档明示支持的取值不做算术校验。"""
|
||||
|
||||
def test_standard_sizes(self):
|
||||
for size in ("1024x1024", "1536x1024", "1024x1536"):
|
||||
validate_size(size)
|
||||
|
||||
def test_auto(self):
|
||||
validate_size("auto")
|
||||
|
||||
def test_none(self):
|
||||
validate_size(None)
|
||||
|
||||
|
||||
class TestEdgeMultipleOf16:
|
||||
"""官方:宽高都必须能被 16 整除。"""
|
||||
|
||||
def test_both_divisible_passes(self):
|
||||
validate_size("1536x864") # 文档给的例子
|
||||
|
||||
def test_width_not_divisible_is_rejected(self):
|
||||
with pytest.raises(ValueError, match="16 整除"):
|
||||
validate_size("1000x864")
|
||||
|
||||
def test_height_not_divisible_is_rejected(self):
|
||||
with pytest.raises(ValueError, match="16 整除"):
|
||||
validate_size("1536x865")
|
||||
|
||||
|
||||
class TestAspectRatioBounds:
|
||||
"""官方:宽高比必须在 1:3 ~ 3:1 之间。"""
|
||||
|
||||
def test_three_to_one_is_allowed(self):
|
||||
# 2880x960 = 3:1,且都能被 16 整除。
|
||||
validate_size("2880x960")
|
||||
|
||||
def test_one_to_three_is_allowed(self):
|
||||
validate_size("960x2880")
|
||||
|
||||
def test_portrait_at_the_pixel_limit_is_allowed(self):
|
||||
"""竖版 4k(2160x3840)与横版同像素量,必须放行。
|
||||
|
||||
按「逐边」理解上限会禁掉所有高于 2160 的竖图,而 1024x1536 是文档明示的
|
||||
标准竖图尺寸——故逐边理解必然过严。见 capabilities 里的说明。
|
||||
"""
|
||||
validate_size("2160x3840")
|
||||
|
||||
def test_wider_than_three_to_one_is_rejected(self):
|
||||
# 3840x960 = 4:1。
|
||||
with pytest.raises(ValueError, match="宽高比"):
|
||||
validate_size("3840x960")
|
||||
|
||||
def test_taller_than_one_to_three_is_rejected(self):
|
||||
# 640x2880 = 1:4.5,且在像素上限之内,故触发的必须是宽高比那条。
|
||||
with pytest.raises(ValueError, match="宽高比"):
|
||||
validate_size("640x2880")
|
||||
|
||||
|
||||
class TestMaximumResolution:
|
||||
def test_the_maximum_itself_is_allowed(self):
|
||||
validate_size("3840x2160")
|
||||
|
||||
def test_longest_edge_beyond_the_cap_is_rejected(self):
|
||||
with pytest.raises(ValueError, match="最长边"):
|
||||
validate_size("4096x2048")
|
||||
|
||||
def test_within_edge_cap_but_beyond_the_pixel_cap_is_rejected(self):
|
||||
# 3840x2688:最长边合规、宽高比合规,但总像素超了。
|
||||
with pytest.raises(ValueError, match="总像素"):
|
||||
validate_size("3840x2688")
|
||||
|
||||
def test_a_tall_portrait_within_both_caps_passes(self):
|
||||
validate_size("1024x2176")
|
||||
|
||||
|
||||
class TestMalformedSize:
|
||||
def test_not_a_size_string(self):
|
||||
with pytest.raises(ValueError, match="形态非法"):
|
||||
validate_size("large")
|
||||
|
||||
def test_missing_separator(self):
|
||||
with pytest.raises(ValueError, match="形态非法"):
|
||||
validate_size("1024")
|
||||
|
||||
def test_parse_returns_none_for_garbage(self):
|
||||
assert parse_size("nonsense") is None
|
||||
|
||||
def test_parse_reads_a_valid_pair(self):
|
||||
size = parse_size("1536x864")
|
||||
assert (size.width, size.height) == (1536, 864)
|
||||
assert str(size) == "1536x864"
|
||||
|
||||
|
||||
class TestQuality:
|
||||
def test_supported_values_pass(self):
|
||||
for quality in ("low", "medium", "high", "auto"):
|
||||
validate_quality(quality)
|
||||
|
||||
def test_xhigh_is_rejected_because_it_is_2_5_only(self):
|
||||
# 官方:xhigh 与 max 仅 gpt-image-2.5 系列支持。
|
||||
with pytest.raises(ValueError, match="xhigh"):
|
||||
validate_quality("xhigh")
|
||||
|
||||
def test_max_is_rejected_too(self):
|
||||
with pytest.raises(ValueError):
|
||||
validate_quality("max")
|
||||
|
||||
def test_none_passes(self):
|
||||
validate_quality(None)
|
||||
|
||||
|
||||
class TestOutputFormat:
|
||||
def test_supported_formats(self):
|
||||
for fmt in ("png", "jpeg", "webp"):
|
||||
validate_output_format(fmt)
|
||||
|
||||
def test_unknown_format_is_rejected(self):
|
||||
with pytest.raises(ValueError, match="非法"):
|
||||
validate_output_format("gif")
|
||||
|
||||
def test_jpeg_with_transparent_background_is_rejected(self):
|
||||
"""jpeg 没有 alpha 通道。
|
||||
|
||||
服务商不会因此报错——只会给一张背景被填成不透明的图,而那是**静默的**
|
||||
不符合预期,比报错难查得多。
|
||||
"""
|
||||
with pytest.raises(ValueError, match="alpha"):
|
||||
validate_output_format("jpeg", transparent=True)
|
||||
|
||||
def test_png_with_transparent_background_is_fine(self):
|
||||
validate_output_format("png", transparent=True)
|
||||
validate_output_format("webp", transparent=True)
|
||||
@@ -0,0 +1,230 @@
|
||||
"""OpenAI HTTP 传输层的测试:鉴权、两种载荷形式、错误解包。
|
||||
|
||||
全部离线(`httpx.MockTransport`),不触网。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from app.domain.generation import Retry, UserFacing
|
||||
from app.providers.openai.client import (
|
||||
DEFAULT_BASE_URL,
|
||||
OpenAiClient,
|
||||
OpenAiError,
|
||||
)
|
||||
|
||||
FAKE_API_KEY = "dummy-api-key-for-tests"
|
||||
|
||||
|
||||
def client_with(handler, **kwargs) -> OpenAiClient:
|
||||
return OpenAiClient(
|
||||
api_key=FAKE_API_KEY,
|
||||
client=httpx.Client(transport=httpx.MockTransport(handler)),
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
class TestCredentialDiscipline:
|
||||
def test_an_empty_key_fails_loudly_at_construction(self):
|
||||
"""缺密钥是配置问题,应当在装配时就说清楚,而不是等第一次调用。"""
|
||||
with pytest.raises(ValueError, match="OPENAI_API_KEY 未配置"):
|
||||
OpenAiClient(api_key="")
|
||||
|
||||
def test_the_key_is_sent_as_a_bearer_token(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["auth"] = request.headers.get("Authorization")
|
||||
return httpx.Response(200, json={"ok": True})
|
||||
|
||||
client_with(handler).post_json("/images/generations", {})
|
||||
assert seen["auth"] == f"Bearer {FAKE_API_KEY}"
|
||||
|
||||
def test_organization_header_is_sent_only_when_configured(self):
|
||||
seen: list = []
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen.append(request.headers.get("OpenAI-Organization"))
|
||||
return httpx.Response(200, json={})
|
||||
|
||||
client_with(handler).post_json("/x", {})
|
||||
client_with(handler, organization="org-abc").post_json("/x", {})
|
||||
assert seen == [None, "org-abc"]
|
||||
|
||||
|
||||
class TestUrlComposition:
|
||||
def test_default_base_url_is_the_official_one(self):
|
||||
assert DEFAULT_BASE_URL == "https://api.openai.com/v1"
|
||||
|
||||
def test_a_trailing_slash_on_the_base_url_does_not_double_up(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["url"] = str(request.url)
|
||||
return httpx.Response(200, json={})
|
||||
|
||||
client_with(handler, base_url="https://proxy.invalid/v1/").post_json("/img", {})
|
||||
assert seen["url"] == "https://proxy.invalid/v1/img"
|
||||
|
||||
|
||||
class TestPayloadForms:
|
||||
def test_post_json_sends_json(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["content_type"] = request.headers.get("Content-Type")
|
||||
seen["body"] = request.content
|
||||
return httpx.Response(200, json={"echo": 1})
|
||||
|
||||
result = client_with(handler).post_json("/x", {"prompt": "hi"})
|
||||
assert seen["content_type"] == "application/json"
|
||||
assert b'"prompt"' in seen["body"]
|
||||
assert result == {"echo": 1}
|
||||
|
||||
def test_post_multipart_repeats_a_field_name(self):
|
||||
"""用 list 而非 dict 传 files 的全部理由:同名字段可以重复。
|
||||
|
||||
编辑端点支持多图,它们共用字段名 `image[]`——dict 表达不了。
|
||||
"""
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["content_type"] = request.headers.get("Content-Type", "")
|
||||
seen["raw"] = request.content
|
||||
return httpx.Response(200, json={})
|
||||
|
||||
client_with(handler).post_multipart(
|
||||
"/images/edits",
|
||||
data={"prompt": "merge"},
|
||||
files=[
|
||||
("image[]", ("a.png", b"first", "image/png")),
|
||||
("image[]", ("b.png", b"second", "image/png")),
|
||||
],
|
||||
)
|
||||
|
||||
assert "multipart/form-data" in seen["content_type"]
|
||||
assert seen["raw"].count(b'name="image[]"') == 2
|
||||
assert b"first" in seen["raw"]
|
||||
assert b"second" in seen["raw"]
|
||||
|
||||
def test_content_type_is_not_hand_written(self):
|
||||
"""multipart 还需要 boundary,手写必错——交给 httpx 按载荷推断。"""
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["ct"] = request.headers.get("Content-Type", "")
|
||||
return httpx.Response(200, json={})
|
||||
|
||||
client_with(handler).post_multipart(
|
||||
"/x", data={}, files=[("image[]", ("a", b"x", "image/png"))]
|
||||
)
|
||||
assert "boundary=" in seen["ct"]
|
||||
|
||||
|
||||
class TestErrorUnwrapping:
|
||||
def test_the_vendor_error_envelope_is_parsed(self):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
429,
|
||||
json={
|
||||
"error": {
|
||||
"message": "Your request rate increased too quickly",
|
||||
"type": "rate_limit_error",
|
||||
"code": "slow_down",
|
||||
"param": None,
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
with pytest.raises(OpenAiError) as caught:
|
||||
client_with(handler).post_json("/x", {})
|
||||
|
||||
error = caught.value
|
||||
assert error.http_status == 429
|
||||
assert error.failure.retry is Retry.BACKOFF
|
||||
assert error.failure.provider_code == "slow_down"
|
||||
assert "increased too quickly" in error.failure.message
|
||||
|
||||
def test_the_param_field_is_extracted_because_classification_needs_it(self):
|
||||
"""`param` 是区分「参数错」与「内容被拒」的唯一结构化线索。
|
||||
|
||||
漏了它,所有 400 都会被当成内容被拒——包括真正的参数错。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
400,
|
||||
json={
|
||||
"error": {
|
||||
"message": "Invalid service_tier argument",
|
||||
"type": "invalid_request_error",
|
||||
"param": "service_tier",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
with pytest.raises(OpenAiError) as caught:
|
||||
client_with(handler).post_json("/x", {})
|
||||
|
||||
# 带 param ⇒ 参数错 ⇒ 服务端错误,而不是让用户去改提示词。
|
||||
assert caught.value.failure.user_facing is UserFacing.SERVER_ERROR
|
||||
|
||||
def test_a_non_json_error_body_still_classifies_by_status(self):
|
||||
"""错误体不一定是文档描述的那个形状(网关、代理会插自己的页面)。
|
||||
|
||||
解析不出时不能静默丢掉线索。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(502, text="<html>bad gateway</html>")
|
||||
|
||||
with pytest.raises(OpenAiError) as caught:
|
||||
client_with(handler).post_json("/x", {})
|
||||
|
||||
assert caught.value.failure.retry is Retry.BACKOFF
|
||||
assert caught.value.failure.provider_code == "HTTP_502"
|
||||
# 原始文本被带上,否则排查时只知道「502」。
|
||||
assert "bad gateway" in caught.value.failure.message
|
||||
|
||||
def test_a_2xx_with_a_non_json_body_is_an_error_not_a_success(self):
|
||||
def handler(request):
|
||||
return httpx.Response(200, text="not json at all")
|
||||
|
||||
with pytest.raises(OpenAiError, match="不是合法 JSON"):
|
||||
client_with(handler).post_json("/x", {})
|
||||
|
||||
def test_a_network_failure_is_classified_as_retryable(self):
|
||||
"""⚠️ 同步接口的超时最危险:那次调用可能已经完成并计费。
|
||||
|
||||
图片接口没有任务 ID 可供事后查询,故这个窗口只能由 max_attempts 兜住。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
raise httpx.ConnectTimeout("timed out")
|
||||
|
||||
with pytest.raises(OpenAiError) as caught:
|
||||
client_with(handler).post_json("/x", {})
|
||||
|
||||
assert caught.value.http_status is None
|
||||
assert caught.value.failure.retry is Retry.BACKOFF
|
||||
assert caught.value.failure.provider_code == "NETWORK_ERROR"
|
||||
|
||||
|
||||
class TestConnectionOwnership:
|
||||
def test_an_injected_client_is_not_closed_by_us(self):
|
||||
"""注入进来的连接池归调用方管理。
|
||||
|
||||
关掉它会让共享同一个 client 的其他适配器一起失效。
|
||||
"""
|
||||
injected = httpx.Client(
|
||||
transport=httpx.MockTransport(lambda r: httpx.Response(200, json={}))
|
||||
)
|
||||
OpenAiClient(api_key=FAKE_API_KEY, client=injected).close()
|
||||
assert not injected.is_closed
|
||||
|
||||
def test_our_own_client_is_closed_on_exit(self):
|
||||
with OpenAiClient(api_key=FAKE_API_KEY) as client:
|
||||
inner = client._client
|
||||
assert inner.is_closed
|
||||
@@ -0,0 +1,211 @@
|
||||
"""OpenAI 错误分类的逐条测试,对照官方 Error codes 页(`doc_temp/gpt错误码.md`)。
|
||||
|
||||
每个用例的码名都**抄自文档**,不是记忆。本文件此前钉的是五个并不存在的码名
|
||||
(见 `errors.py` 顶部),重写后每条都能在文档里搜到。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from app.domain.generation import Retry, UserFacing
|
||||
from app.providers.openai.errors import classify
|
||||
|
||||
|
||||
class TestBillingAndQuotaCodes:
|
||||
"""官方表 429 的六种里有四种是钱的问题,且文档明说重试不会恢复访问。"""
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"code",
|
||||
[
|
||||
"credit_balance_exhausted",
|
||||
"organization_spend_limit_exceeded",
|
||||
"project_spend_limit_exceeded",
|
||||
"organization_usage_limit_exceeded",
|
||||
],
|
||||
)
|
||||
def test_they_are_all_quota_not_backoff(self, code):
|
||||
failure = classify(429, error_code=code)
|
||||
# Retry.QUOTA 的语义正是文档那句「Retrying billing, spend, or quota errors
|
||||
# won't restore API access」——Worker 不该把重试预算花在这上面。
|
||||
assert failure.retry is Retry.QUOTA
|
||||
assert failure.user_facing is UserFacing.QUOTA_EXHAUSTED
|
||||
assert failure.provider_code == code
|
||||
|
||||
def test_the_broad_type_is_recognised_when_the_code_is_absent(self):
|
||||
"""文档:「The broader `error.type` can still be `insufficient_quota`」。"""
|
||||
failure = classify(429, error_type="insufficient_quota")
|
||||
assert failure.retry is Retry.QUOTA
|
||||
|
||||
def test_a_known_code_wins_over_the_broad_type(self):
|
||||
# code 比 type 具体;此例两者同向,测的是优先级本身被实现了。
|
||||
failure = classify(
|
||||
429, error_type="insufficient_quota", error_code="credit_balance_exhausted"
|
||||
)
|
||||
assert failure.provider_code == "credit_balance_exhausted"
|
||||
assert failure.retry is Retry.QUOTA
|
||||
|
||||
@pytest.mark.parametrize("field", ["code", "type"])
|
||||
@pytest.mark.parametrize(
|
||||
"token", ["insufficient_quota", "credit_balance_exhausted", "slow_down"]
|
||||
)
|
||||
def test_a_documented_token_is_honoured_in_either_field(self, field, token):
|
||||
"""同一张表按 `code` 与 `type` 两个字段分别查。
|
||||
|
||||
文档把某个标识列在哪个字段下,不保证实际响应里它一定出现在那个字段。查错
|
||||
字段的代价是把配额耗尽降级成限流、白烧一轮重试预算;而两边都查不引入任何
|
||||
**新**词汇——表里每个键都抄自文档。
|
||||
"""
|
||||
expected = Retry.BACKOFF if token == "slow_down" else Retry.QUOTA
|
||||
assert classify(429, **{f"error_{field}": token}).retry is expected
|
||||
|
||||
|
||||
class TestRateLimitCodes:
|
||||
def test_slow_down_backs_off(self):
|
||||
"""`slow_down` 是「涨得太快」,文档说即使在 RPM/TPM 额度内也会触发。"""
|
||||
failure = classify(429, error_type="rate_limit_error", error_code="slow_down")
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.RATE_LIMITED
|
||||
|
||||
def test_the_rate_limit_type_alone_is_enough(self):
|
||||
failure = classify(429, error_type="rate_limit_error")
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.RATE_LIMITED
|
||||
|
||||
def test_a_bare_429_defaults_to_rate_limiting(self):
|
||||
"""这个默认值现在是**文档支持的**,不再是猜测。
|
||||
|
||||
官方表 429 的六种里,只有「Rate limit reached for requests」没有文档化的
|
||||
code——故一个既无 code 又无已知 type 的 429 就是它。
|
||||
"""
|
||||
failure = classify(429)
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.RATE_LIMITED
|
||||
|
||||
|
||||
class TestOverload:
|
||||
def test_server_is_overloaded_is_retryable(self):
|
||||
failure = classify(
|
||||
503,
|
||||
error_type="service_unavailable_error",
|
||||
error_code="server_is_overloaded",
|
||||
)
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.UPSTREAM_UNAVAILABLE
|
||||
|
||||
def test_the_vendors_429_to_503_reclassification_does_not_affect_us(self):
|
||||
"""文档提醒:视频过载已从 429 改判为 503,只捕一类会漏。
|
||||
|
||||
我们两者都归可重试,故这次改判对本服务无影响。这条测试就是那个「无影响」
|
||||
的凭据——哪天有人把 5xx 改成不可重试,它会红。
|
||||
"""
|
||||
assert classify(429).retry is Retry.BACKOFF
|
||||
assert classify(503).retry is Retry.BACKOFF
|
||||
|
||||
|
||||
class TestBadRequestIsTwoThingsAtOnce:
|
||||
"""400 既是参数错也是(推断的)内容被拒,用 `error.param` 区分。
|
||||
|
||||
官方错误码表里**没有任何内容审核/侵权码**,故内容拒绝只能以通用 400 到达。
|
||||
"""
|
||||
|
||||
def test_a_param_error_is_our_problem_not_the_prompts(self):
|
||||
# 文档的 service_tier 那条明确说会带 param。
|
||||
failure = classify(
|
||||
400,
|
||||
error_type="invalid_request_error",
|
||||
error_param="service_tier",
|
||||
message="Invalid service_tier argument",
|
||||
)
|
||||
assert failure.retry is Retry.NEVER
|
||||
assert failure.user_facing is UserFacing.SERVER_ERROR
|
||||
|
||||
def test_a_400_without_param_is_treated_as_a_content_refusal(self):
|
||||
"""推断分支:不指向任何参数的 400,按内容被拒处理。
|
||||
|
||||
若归「服务端错误」,用户会看到「服务出错了」并反复重试同一个注定被拒的
|
||||
提示词——那是更糟的结果。
|
||||
"""
|
||||
failure = classify(400, message="Your request was rejected")
|
||||
assert failure.retry is Retry.NEVER
|
||||
assert failure.user_facing is UserFacing.CONTENT_REJECTED
|
||||
|
||||
def test_the_inferred_branch_logs_a_warning(self, caplog):
|
||||
"""这条分支是推断的,日志是日后校正它的唯一素材来源。"""
|
||||
with caplog.at_level("WARNING"):
|
||||
classify(400, message="something was refused")
|
||||
assert "推断" in caplog.text
|
||||
|
||||
def test_an_empty_param_counts_as_absent(self):
|
||||
# 厂商可能回 "" 或 null,两者语义相同。
|
||||
assert classify(400, error_param="").user_facing is UserFacing.CONTENT_REJECTED
|
||||
|
||||
|
||||
class TestAuthAndGeo:
|
||||
def test_invalid_authentication_is_terminal_and_not_the_users_fault(self):
|
||||
failure = classify(401, message="Incorrect API key provided")
|
||||
assert failure.retry is Retry.NEVER
|
||||
assert failure.user_facing is UserFacing.SERVER_ERROR
|
||||
|
||||
def test_unsupported_country_is_also_our_configuration_problem(self):
|
||||
failure = classify(403, message="Country, region, or territory not supported")
|
||||
assert failure.retry is Retry.NEVER
|
||||
assert failure.user_facing is UserFacing.SERVER_ERROR
|
||||
|
||||
|
||||
class TestStatusCodeFallback:
|
||||
@pytest.mark.parametrize("status", [404, 409, 422])
|
||||
def test_other_4xx_are_terminal(self, status):
|
||||
assert classify(status).retry is Retry.NEVER
|
||||
|
||||
@pytest.mark.parametrize("status", [500, 502, 504])
|
||||
def test_5xx_is_retryable(self, status):
|
||||
failure = classify(status)
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.UPSTREAM_UNAVAILABLE
|
||||
|
||||
def test_network_failure_is_retryable(self):
|
||||
failure = classify(None, message="connect timeout")
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
assert failure.user_facing is UserFacing.UPSTREAM_UNAVAILABLE
|
||||
|
||||
def test_an_unknown_status_leans_towards_retrying(self):
|
||||
assert classify(599).retry is Retry.BACKOFF
|
||||
|
||||
|
||||
class TestUnknownCodesFallThroughRatherThanBlowUp:
|
||||
"""厂商随时可能加新码;不认识的码必须回落,不能抛异常或判死。"""
|
||||
|
||||
def test_an_unknown_code_on_a_429_still_backs_off(self):
|
||||
failure = classify(429, error_code="some_future_code")
|
||||
assert failure.retry is Retry.BACKOFF
|
||||
# 码本身照原样带出去——它是排查这条新码的唯一线索。
|
||||
assert failure.provider_code == "some_future_code"
|
||||
|
||||
def test_an_unknown_type_on_a_5xx_still_backs_off(self):
|
||||
assert classify(500, error_type="brand_new_error").retry is Retry.BACKOFF
|
||||
|
||||
|
||||
class TestProviderCodeIsNeverEmpty:
|
||||
"""它是排查时的第一线索,空字符串等于把线索丢了。"""
|
||||
|
||||
def test_code_is_preferred(self):
|
||||
assert classify(400, error_type="t", error_code="c").provider_code == "c"
|
||||
|
||||
def test_type_is_used_when_code_is_absent(self):
|
||||
assert classify(400, error_type="t").provider_code == "t"
|
||||
|
||||
def test_status_label_is_the_last_resort(self):
|
||||
assert classify(503).provider_code == "HTTP_503"
|
||||
|
||||
def test_network_failures_get_a_named_label_too(self):
|
||||
assert classify(None).provider_code == "NETWORK_ERROR"
|
||||
|
||||
|
||||
class TestMessageIsCarriedThrough:
|
||||
def test_message_is_kept(self):
|
||||
assert classify(400, message="bad size").message == "bad size"
|
||||
|
||||
def test_missing_message_becomes_empty_not_none(self):
|
||||
# 契约里 message 是字符串;None 会在序列化处变成 null 而非 ""。
|
||||
assert classify(400).message == ""
|
||||
@@ -0,0 +1,433 @@
|
||||
"""gpt-image-2 适配器的测试:文生图、图生图、响应映射。
|
||||
|
||||
全部离线(`httpx.MockTransport`),并用同步的注册表配置(`max_workers=1`)+ 轮询
|
||||
等待,使断言不依赖时序。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
import time
|
||||
|
||||
import httpx
|
||||
import pytest
|
||||
|
||||
from app.domain.generation import (
|
||||
GenerationRequest,
|
||||
MediaKind,
|
||||
Retry,
|
||||
TaskState,
|
||||
UserFacing,
|
||||
)
|
||||
from app.providers.openai.client import OpenAiClient
|
||||
from app.providers.openai.image import PROVIDER_CODE, OpenAiImageProvider
|
||||
from app.providers.openai.tasks import SyncTaskRegistry
|
||||
|
||||
FAKE_API_KEY = "dummy-api-key-for-tests"
|
||||
MODEL = "gpt-image-2"
|
||||
PNG_B64 = base64.b64encode(b"fake-png-bytes").decode()
|
||||
|
||||
#: 官方文档「Generate image」一节的响应形状。
|
||||
OFFICIAL_RESPONSE = {
|
||||
"created": 1713833628,
|
||||
"data": [{"b64_json": PNG_B64}],
|
||||
"size": "1024x1024",
|
||||
"output_format": "png",
|
||||
"quality": "high",
|
||||
"usage": {
|
||||
"total_tokens": 100,
|
||||
"input_tokens": 50,
|
||||
"output_tokens": 50,
|
||||
"input_tokens_details": {"text_tokens": 10, "image_tokens": 40},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
def provider_with(handler, *, fetch_handler=None, **kwargs) -> OpenAiImageProvider:
|
||||
client = OpenAiClient(
|
||||
api_key=FAKE_API_KEY,
|
||||
base_url="https://api.openai.invalid/v1",
|
||||
client=httpx.Client(transport=httpx.MockTransport(handler)),
|
||||
)
|
||||
fetch_client = (
|
||||
httpx.Client(transport=httpx.MockTransport(fetch_handler))
|
||||
if fetch_handler
|
||||
else None
|
||||
)
|
||||
return OpenAiImageProvider(
|
||||
client,
|
||||
SyncTaskRegistry(max_workers=1, max_entries=8),
|
||||
input_fetch_client=fetch_client,
|
||||
**kwargs,
|
||||
)
|
||||
|
||||
|
||||
def image_request(**kwargs) -> GenerationRequest:
|
||||
defaults = {
|
||||
"job_id": "job-1",
|
||||
"media_kind": MediaKind.IMAGE,
|
||||
"model": MODEL,
|
||||
"prompt": "一只柯基在草地上",
|
||||
}
|
||||
return GenerationRequest(**{**defaults, **kwargs})
|
||||
|
||||
|
||||
def await_result(provider: OpenAiImageProvider, task_id: str, timeout: float = 3.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
result = provider.fetch(task_id)
|
||||
if result.state.is_terminal:
|
||||
return result
|
||||
time.sleep(0.01)
|
||||
raise AssertionError("任务未在超时内结束")
|
||||
|
||||
|
||||
class TestIdentity:
|
||||
def test_provider_code_matches_the_catalog_column(self):
|
||||
assert (
|
||||
provider_with(lambda r: httpx.Response(200, json={})).provider_code
|
||||
== PROVIDER_CODE
|
||||
)
|
||||
|
||||
def test_declares_itself_as_an_image_provider(self):
|
||||
assert (
|
||||
provider_with(lambda r: httpx.Response(200, json={})).media_kind
|
||||
is MediaKind.IMAGE
|
||||
)
|
||||
|
||||
def test_rejects_a_misrouted_video_request(self):
|
||||
with pytest.raises(ValueError, match="只处理图片生成"):
|
||||
provider_with(lambda r: httpx.Response(200, json={})).submit(
|
||||
image_request(media_kind=MediaKind.VIDEO)
|
||||
)
|
||||
|
||||
|
||||
class TestSubmitIsNonBlockingDespiteTheSyncVendor:
|
||||
"""整个 tasks 层存在的理由:服务商同步,但我们对外必须是异步的。"""
|
||||
|
||||
def test_submit_returns_a_task_id_without_waiting_for_the_image(self):
|
||||
import threading
|
||||
|
||||
release = threading.Event()
|
||||
|
||||
def handler(request):
|
||||
release.wait(timeout=3)
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler)
|
||||
begin = time.monotonic()
|
||||
task_id = provider.submit(image_request())
|
||||
elapsed = time.monotonic() - begin
|
||||
|
||||
assert task_id.startswith("local-")
|
||||
assert elapsed < 0.5
|
||||
assert provider.fetch(task_id).state is TaskState.RUNNING
|
||||
release.set()
|
||||
await_result(provider, task_id)
|
||||
|
||||
|
||||
class TestTextToImage:
|
||||
def test_posts_json_to_the_generations_endpoint(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
seen["path"] = request.url.path
|
||||
seen["body"] = json.loads(request.content)
|
||||
seen["auth"] = request.headers.get("Authorization")
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler)
|
||||
await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert seen["path"].endswith("/images/generations")
|
||||
assert seen["auth"] == f"Bearer {FAKE_API_KEY}"
|
||||
assert seen["body"]["prompt"] == "一只柯基在草地上"
|
||||
assert seen["body"]["model"] == MODEL
|
||||
# 一个任务一个产物:generation_jobs.output_asset_id 是单值。
|
||||
assert seen["body"]["n"] == 1
|
||||
|
||||
def test_pixel_size_is_sent_only_when_both_dimensions_are_given(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
seen.update(json.loads(request.content))
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler)
|
||||
await_result(
|
||||
provider, provider.submit(image_request(width_px=1536, height_px=864))
|
||||
)
|
||||
assert seen["size"] == "1536x864"
|
||||
|
||||
def test_half_a_size_is_treated_as_no_size_at_all(self):
|
||||
# 只给宽不给高时交给模型自选,而不是猜一个高度出来。
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
seen.update(json.loads(request.content))
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler)
|
||||
await_result(provider, provider.submit(image_request(width_px=1024)))
|
||||
assert "size" not in seen
|
||||
|
||||
def test_safety_identifier_is_sent_as_the_vendors_user_field(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
seen.update(json.loads(request.content))
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler)
|
||||
await_result(
|
||||
provider, provider.submit(image_request(safety_identifier="abc123hash"))
|
||||
)
|
||||
# 厂商字段名是 user,不是 safety_identifier。
|
||||
assert seen["user"] == "abc123hash"
|
||||
assert "safety_identifier" not in seen
|
||||
|
||||
|
||||
class TestModelResolution:
|
||||
def test_request_model_wins_over_the_configured_default(self):
|
||||
seen: dict = {}
|
||||
|
||||
def handler(request: httpx.Request) -> httpx.Response:
|
||||
import json
|
||||
|
||||
seen.update(json.loads(request.content))
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
provider = provider_with(handler, default_model="fallback")
|
||||
await_result(provider, provider.submit(image_request(model="snapshot-model")))
|
||||
assert seen["model"] == "snapshot-model"
|
||||
|
||||
def test_no_model_anywhere_fails_before_any_call(self):
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
with pytest.raises(ValueError, match="未指定模型"):
|
||||
provider_with(handler).submit(image_request(model=""))
|
||||
|
||||
|
||||
class TestLocalValidationHappensBeforeAccepting:
|
||||
"""违规请求应当立刻被拒,而不是变成一个注定失败的后台任务。
|
||||
|
||||
后者要等一轮轮询才被发现,还白占一个线程池槽位。
|
||||
"""
|
||||
|
||||
def test_bad_size_is_rejected_synchronously(self):
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
with pytest.raises(ValueError, match="16 整除"):
|
||||
provider_with(handler).submit(image_request(width_px=1000, height_px=864))
|
||||
|
||||
|
||||
class TestImageToImage:
|
||||
def test_downloads_the_input_then_posts_multipart_to_edits(self):
|
||||
seen: dict = {}
|
||||
|
||||
def openai_handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["path"] = request.url.path
|
||||
seen["content_type"] = request.headers.get("Content-Type", "")
|
||||
seen["raw"] = request.content
|
||||
return httpx.Response(200, json=OFFICIAL_RESPONSE)
|
||||
|
||||
def fetch_handler(request: httpx.Request) -> httpx.Response:
|
||||
seen["fetched"] = str(request.url)
|
||||
return httpx.Response(
|
||||
200,
|
||||
content=b"reference-image-bytes",
|
||||
headers={"Content-Type": "image/png"},
|
||||
)
|
||||
|
||||
provider = provider_with(openai_handler, fetch_handler=fetch_handler)
|
||||
await_result(
|
||||
provider, provider.submit(image_request(input_url="https://x/ref.png"))
|
||||
)
|
||||
|
||||
assert seen["fetched"] == "https://x/ref.png"
|
||||
# 编辑接口走 multipart 而非 JSON——`image` 是文件。
|
||||
assert seen["path"].endswith("/images/edits")
|
||||
assert "multipart/form-data" in seen["content_type"]
|
||||
assert b"reference-image-bytes" in seen["raw"]
|
||||
# 字段名统一用 image[](该接口支持多图且共用此名),避免单图多图两种写法。
|
||||
assert b'name="image[]"' in seen["raw"]
|
||||
|
||||
def test_an_undownloadable_input_is_rejected_synchronously(self):
|
||||
"""取不到参考图就是参数问题,应立刻拒绝而不是变成后台失败。"""
|
||||
|
||||
def openai_handler(request):
|
||||
raise AssertionError("不应发出生成请求")
|
||||
|
||||
def fetch_handler(request):
|
||||
raise httpx.ConnectError("refused")
|
||||
|
||||
with pytest.raises(ValueError, match="无法下载"):
|
||||
provider_with(openai_handler, fetch_handler=fetch_handler).submit(
|
||||
image_request(input_url="https://unreachable/ref.png")
|
||||
)
|
||||
|
||||
def test_an_oversized_input_is_rejected_by_its_declared_length(self):
|
||||
"""第一道:声明的 Content-Length 就超了,连读都不必读。"""
|
||||
|
||||
def openai_handler(request):
|
||||
raise AssertionError("不应发出生成请求")
|
||||
|
||||
def fetch_handler(request):
|
||||
return httpx.Response(200, content=b"x" * (21 * 1024 * 1024))
|
||||
|
||||
with pytest.raises(ValueError, match="参考图过大"):
|
||||
provider_with(openai_handler, fetch_handler=fetch_handler).submit(
|
||||
image_request(input_url="https://x/huge.png")
|
||||
)
|
||||
|
||||
def test_an_oversized_input_is_caught_even_when_the_length_lies(self):
|
||||
"""第二道:不信 Content-Length——它可能缺失或撒谎,必须边读边数。
|
||||
|
||||
用分块流式响应(无 Content-Length)构造这种情形:只有累计计数才拦得住。
|
||||
"""
|
||||
|
||||
def openai_handler(request):
|
||||
raise AssertionError("不应发出生成请求")
|
||||
|
||||
def chunks():
|
||||
# 每块 1MB,共 21MB,分块传输故无 Content-Length。
|
||||
for _ in range(21):
|
||||
yield b"x" * (1024 * 1024)
|
||||
|
||||
def fetch_handler(request):
|
||||
return httpx.Response(200, content=chunks())
|
||||
|
||||
with pytest.raises(ValueError, match="超过上限"):
|
||||
provider_with(openai_handler, fetch_handler=fetch_handler).submit(
|
||||
image_request(input_url="https://x/lying.png")
|
||||
)
|
||||
|
||||
|
||||
class TestResponseMapping:
|
||||
def test_dimensions_come_straight_from_the_response(self):
|
||||
"""不像火山那边要查对照表——响应直接给 size。"""
|
||||
provider = provider_with(lambda r: httpx.Response(200, json=OFFICIAL_RESPONSE))
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.state is TaskState.SUCCEEDED
|
||||
assert (result.width_px, result.height_px) == (1024, 1024)
|
||||
assert result.output_format == "png"
|
||||
assert result.billing_tokens == 100
|
||||
assert result.failure is None
|
||||
|
||||
def test_arbitrary_size_is_parsed_too(self):
|
||||
payload = {**OFFICIAL_RESPONSE, "size": "1536x864"}
|
||||
provider = provider_with(lambda r: httpx.Response(200, json=payload))
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
assert (result.width_px, result.height_px) == (1536, 864)
|
||||
|
||||
def test_asset_url_is_a_data_uri_with_no_expiry(self):
|
||||
"""产物是 base64 字节,不是可访问 URL。
|
||||
|
||||
没有「24 小时过期」问题(与火山不同),故 expires_in 为 None 而不是编一个
|
||||
假的秒数。但它**也不是可长期保存的地址**——绝不能原样写进 external_url。
|
||||
"""
|
||||
provider = provider_with(lambda r: httpx.Response(200, json=OFFICIAL_RESPONSE))
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.asset_url.startswith("data:image/png;base64,")
|
||||
assert PNG_B64 in result.asset_url
|
||||
assert result.asset_url_expires_in_sec is None
|
||||
|
||||
def test_a_2xx_without_image_data_is_a_retryable_failure(self):
|
||||
"""服务商说成功却没给图——那不是成功。"""
|
||||
provider = provider_with(
|
||||
lambda r: httpx.Response(200, json={"created": 1, "data": []})
|
||||
)
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.state is TaskState.FAILED
|
||||
assert result.failure.retry is Retry.BACKOFF
|
||||
assert result.failure.provider_code == "MissingImageData"
|
||||
|
||||
|
||||
class TestFailurePropagation:
|
||||
"""只验「厂商错误能原样穿过后台线程到达查询结果」,分类本身在 test_errors.py。
|
||||
|
||||
码名全部抄自官方 Error codes 页(`doc_temp/gpt错误码.md`)。
|
||||
"""
|
||||
|
||||
def test_vendor_rate_limit_becomes_a_retryable_failure(self):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
429,
|
||||
json={
|
||||
"error": {
|
||||
"message": "Your request rate increased too quickly",
|
||||
"type": "rate_limit_error",
|
||||
"code": "slow_down",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
provider = provider_with(handler)
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.state is TaskState.FAILED
|
||||
assert result.failure.retry is Retry.BACKOFF
|
||||
assert result.failure.user_facing is UserFacing.RATE_LIMITED
|
||||
|
||||
def test_an_exhausted_credit_balance_is_not_worth_retrying(self):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
429,
|
||||
json={
|
||||
"error": {
|
||||
"message": "Your organization has no prepaid credits remaining",
|
||||
"type": "insufficient_quota",
|
||||
"code": "credit_balance_exhausted",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
provider = provider_with(handler)
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.failure.retry is Retry.QUOTA
|
||||
assert result.failure.user_facing is UserFacing.QUOTA_EXHAUSTED
|
||||
|
||||
def test_a_bare_400_is_surfaced_as_a_content_refusal(self):
|
||||
"""官方错误码表里**没有任何内容审核/侵权码**。
|
||||
|
||||
故一次「这个提示词不能生成」只能以通用 400 到达。不带 `error.param` 时按
|
||||
内容被拒处理——详见 `errors.py` 的 `_classify_bad_request`。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
400,
|
||||
json={
|
||||
"error": {
|
||||
"message": "Your request was rejected",
|
||||
"type": "invalid_request_error",
|
||||
}
|
||||
},
|
||||
)
|
||||
|
||||
provider = provider_with(handler)
|
||||
result = await_result(provider, provider.submit(image_request()))
|
||||
|
||||
assert result.failure.retry is Retry.NEVER
|
||||
assert result.failure.user_facing is UserFacing.CONTENT_REJECTED
|
||||
|
||||
|
||||
class TestUnknownTaskId:
|
||||
def test_fetch_raises_key_error_so_the_endpoint_can_map_it(self):
|
||||
provider = provider_with(lambda r: httpx.Response(200, json={}))
|
||||
with pytest.raises(KeyError):
|
||||
provider.fetch("local-never-existed")
|
||||
@@ -0,0 +1,249 @@
|
||||
"""同步→异步任务注册表的测试。
|
||||
|
||||
这一层是 gpt-image-2 与 Seedance 的根本差异所在(同步接口没有任务 ID),也是唯一
|
||||
让本服务**持有状态**的地方。它写错的表现是丢任务或内存泄漏,两者都要到生产才显形,
|
||||
所以逐条钉住。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
|
||||
import pytest
|
||||
|
||||
from app.domain.generation import (
|
||||
Failure,
|
||||
GenerationResult,
|
||||
Retry,
|
||||
TaskState,
|
||||
UserFacing,
|
||||
)
|
||||
from app.providers.openai.client import OpenAiError
|
||||
from app.providers.openai.tasks import LOCAL_TASK_PREFIX, SyncTaskRegistry
|
||||
|
||||
|
||||
def succeeded(url: str = "data:image/png;base64,AAA") -> GenerationResult:
|
||||
return GenerationResult(
|
||||
provider_request_id="", state=TaskState.SUCCEEDED, progress=100, asset_url=url
|
||||
)
|
||||
|
||||
|
||||
def wait_until_done(registry: SyncTaskRegistry, task_id: str, timeout: float = 3.0):
|
||||
deadline = time.monotonic() + timeout
|
||||
while time.monotonic() < deadline:
|
||||
result = registry.fetch(task_id)
|
||||
if result is not None and result.state.is_terminal:
|
||||
return result
|
||||
time.sleep(0.01)
|
||||
raise AssertionError(f"任务 {task_id} 在 {timeout}s 内未结束")
|
||||
|
||||
|
||||
class TestSubmitReturnsImmediately:
|
||||
def test_submit_does_not_block_on_the_slow_call(self):
|
||||
"""这是整层存在的理由:同步调用必须在后台跑,提交立刻返回。"""
|
||||
registry = SyncTaskRegistry(max_workers=2)
|
||||
started = threading.Event()
|
||||
release = threading.Event()
|
||||
|
||||
def slow():
|
||||
started.set()
|
||||
release.wait(timeout=3)
|
||||
return succeeded()
|
||||
|
||||
begin = time.monotonic()
|
||||
task_id = registry.submit(slow)
|
||||
elapsed = time.monotonic() - begin
|
||||
|
||||
assert started.wait(timeout=2)
|
||||
# 提交本身几乎不耗时,尽管那个调用还在跑。
|
||||
assert elapsed < 0.5
|
||||
assert registry.fetch(task_id).state is TaskState.RUNNING
|
||||
release.set()
|
||||
wait_until_done(registry, task_id)
|
||||
registry.shutdown()
|
||||
|
||||
def test_task_id_is_prefixed_so_logs_distinguish_it_from_a_vendor_id(self):
|
||||
registry = SyncTaskRegistry()
|
||||
task_id = registry.submit(succeeded)
|
||||
assert task_id.startswith(LOCAL_TASK_PREFIX)
|
||||
registry.shutdown()
|
||||
|
||||
def test_ids_are_unique(self):
|
||||
registry = SyncTaskRegistry()
|
||||
ids = {registry.submit(succeeded) for _ in range(20)}
|
||||
assert len(ids) == 20
|
||||
registry.shutdown()
|
||||
|
||||
|
||||
class TestInFlightAndCompletedStates:
|
||||
def test_running_reports_zero_progress(self):
|
||||
# 同步接口不提供进度,只能报 0。契约本就不承诺平滑进度(ADR-023 §10)。
|
||||
registry = SyncTaskRegistry()
|
||||
release = threading.Event()
|
||||
task_id = registry.submit(lambda: (release.wait(timeout=3), succeeded())[1])
|
||||
|
||||
result = registry.fetch(task_id)
|
||||
assert result.state is TaskState.RUNNING
|
||||
assert result.progress == 0
|
||||
release.set()
|
||||
registry.shutdown()
|
||||
|
||||
def test_completed_result_is_returned_with_the_real_task_id(self):
|
||||
"""工作函数拿不到自己的任务 ID,故由注册表在 fetch 时填入。"""
|
||||
registry = SyncTaskRegistry()
|
||||
task_id = registry.submit(succeeded)
|
||||
|
||||
result = wait_until_done(registry, task_id)
|
||||
assert result.state is TaskState.SUCCEEDED
|
||||
assert result.provider_request_id == task_id
|
||||
assert result.asset_url is not None
|
||||
registry.shutdown()
|
||||
|
||||
def test_result_can_be_fetched_more_than_once(self):
|
||||
# Java 侧可能重复轮询同一个已完成任务(比如一次响应丢了)。
|
||||
registry = SyncTaskRegistry()
|
||||
task_id = registry.submit(succeeded)
|
||||
wait_until_done(registry, task_id)
|
||||
|
||||
assert registry.fetch(task_id).state is TaskState.SUCCEEDED
|
||||
assert registry.fetch(task_id).state is TaskState.SUCCEEDED
|
||||
registry.shutdown()
|
||||
|
||||
|
||||
class TestUnknownIdIsIndistinguishableOnPurpose:
|
||||
def test_never_existed_returns_none(self):
|
||||
"""从未存在过与「进程重启后丢失」无法区分——不假装区分。
|
||||
|
||||
两种情形对调用方的处置相同:停止轮询、任务判失败。
|
||||
"""
|
||||
assert SyncTaskRegistry().fetch("local-nonexistent") is None
|
||||
|
||||
|
||||
class TestFailurePropagation:
|
||||
def test_a_classified_provider_error_keeps_its_classification(self):
|
||||
registry = SyncTaskRegistry()
|
||||
failure = Failure(
|
||||
Retry.QUOTA, UserFacing.QUOTA_EXHAUSTED, "insufficient_quota", "no credit"
|
||||
)
|
||||
|
||||
def failing():
|
||||
raise OpenAiError(failure, 429)
|
||||
|
||||
task_id = registry.submit(failing)
|
||||
result = wait_until_done(registry, task_id)
|
||||
|
||||
assert result.state is TaskState.FAILED
|
||||
assert result.failure.retry is Retry.QUOTA
|
||||
assert result.failure.provider_code == "insufficient_quota"
|
||||
registry.shutdown()
|
||||
|
||||
def test_an_unclassified_exception_becomes_a_retryable_failure_naming_its_type(
|
||||
self,
|
||||
):
|
||||
"""未分类异常不该让任务卡住,也不该丢掉线索。
|
||||
|
||||
把异常类型名放进 provider_code——否则排查时只有一句空洞的错误消息。
|
||||
"""
|
||||
registry = SyncTaskRegistry()
|
||||
|
||||
def boom():
|
||||
raise ZeroDivisionError("unexpected")
|
||||
|
||||
task_id = registry.submit(boom)
|
||||
result = wait_until_done(registry, task_id)
|
||||
|
||||
assert result.state is TaskState.FAILED
|
||||
assert result.failure.retry is Retry.BACKOFF
|
||||
assert result.failure.provider_code == "ZeroDivisionError"
|
||||
registry.shutdown()
|
||||
|
||||
|
||||
class TestCapacityIsBounded:
|
||||
def test_submitting_beyond_the_cap_is_rejected_rather_than_evicting(self):
|
||||
"""满了要拒绝新提交,**不能挤掉在途任务**。
|
||||
|
||||
挤掉一个在途任务会让一个可能正在计费的调用变成无人认领的开销。
|
||||
"""
|
||||
registry = SyncTaskRegistry(max_workers=1, max_entries=2)
|
||||
release = threading.Event()
|
||||
|
||||
def blocked():
|
||||
release.wait(timeout=3)
|
||||
return succeeded()
|
||||
|
||||
registry.submit(blocked)
|
||||
registry.submit(blocked)
|
||||
|
||||
with pytest.raises(RuntimeError, match="上限"):
|
||||
registry.submit(blocked)
|
||||
|
||||
release.set()
|
||||
registry.shutdown()
|
||||
|
||||
def test_concurrency_is_capped_by_max_workers(self):
|
||||
"""并发上限同时也是「同时有多少钱在花」的上界。"""
|
||||
registry = SyncTaskRegistry(max_workers=2, max_entries=10)
|
||||
concurrent = []
|
||||
lock = threading.Lock()
|
||||
release = threading.Event()
|
||||
|
||||
def work():
|
||||
with lock:
|
||||
concurrent.append(1)
|
||||
peak = len(concurrent)
|
||||
release.wait(timeout=3)
|
||||
with lock:
|
||||
concurrent.pop()
|
||||
return succeeded(f"data:image/png;base64,{peak}")
|
||||
|
||||
for _ in range(5):
|
||||
registry.submit(work)
|
||||
time.sleep(0.2)
|
||||
|
||||
with lock:
|
||||
assert len(concurrent) <= 2
|
||||
|
||||
release.set()
|
||||
registry.shutdown()
|
||||
|
||||
|
||||
class TestExpiryOnlyTouchesCompletedEntries:
|
||||
def test_completed_entries_expire(self):
|
||||
registry = SyncTaskRegistry(completed_ttl_sec=0.05, max_entries=10)
|
||||
task_id = registry.submit(succeeded)
|
||||
wait_until_done(registry, task_id)
|
||||
time.sleep(0.1)
|
||||
|
||||
# 清理发生在下一次 submit 时(不额外起一个定时线程——本服务的规模用不上)。
|
||||
registry.submit(succeeded)
|
||||
assert registry.fetch(task_id) is None
|
||||
registry.shutdown()
|
||||
|
||||
def test_in_flight_entries_are_never_expired(self):
|
||||
"""在途任务无论多久都不清——它可能正在花钱。"""
|
||||
registry = SyncTaskRegistry(completed_ttl_sec=0.01, max_entries=10)
|
||||
release = threading.Event()
|
||||
task_id = registry.submit(lambda: (release.wait(timeout=3), succeeded())[1])
|
||||
time.sleep(0.1)
|
||||
|
||||
registry.submit(succeeded)
|
||||
# 仍在注册表里。
|
||||
assert registry.fetch(task_id) is not None
|
||||
assert registry.fetch(task_id).state is TaskState.RUNNING
|
||||
|
||||
release.set()
|
||||
registry.shutdown()
|
||||
|
||||
def test_ttl_starts_at_completion_not_at_creation(self):
|
||||
"""否则一个耗时接近 TTL 的任务会在刚完成时就被清掉——结果还没被取走。"""
|
||||
registry = SyncTaskRegistry(completed_ttl_sec=5.0, max_entries=10)
|
||||
release = threading.Event()
|
||||
task_id = registry.submit(lambda: (release.wait(timeout=3), succeeded())[1])
|
||||
time.sleep(0.15)
|
||||
release.set()
|
||||
wait_until_done(registry, task_id)
|
||||
|
||||
registry.submit(succeeded)
|
||||
assert registry.fetch(task_id).state is TaskState.SUCCEEDED
|
||||
registry.shutdown()
|
||||
@@ -0,0 +1,130 @@
|
||||
"""模型限制表的测试。
|
||||
|
||||
每个用例对应官方文档里两个已接入模型的一条具体限制。这些限制若搞错,表现是
|
||||
用户提交后拿到一个失败任务(厂商 400),而本地测试全绿——所以值得逐条钉住。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import pytest
|
||||
|
||||
from app.providers.volcengine.capabilities import (
|
||||
AUTO_DURATION,
|
||||
SEEDANCE_2_0,
|
||||
SEEDANCE_2_5,
|
||||
limits_for,
|
||||
validate,
|
||||
)
|
||||
|
||||
#: 账号实际接入的两个模型 ID。
|
||||
MODEL_2_5 = "doubao-seedance-2-5-260628"
|
||||
MODEL_2_0 = "doubao-seedance-2-0-260128"
|
||||
|
||||
|
||||
class TestFamilyDetection:
|
||||
def test_both_activated_models_are_recognised(self):
|
||||
assert limits_for(MODEL_2_5) is SEEDANCE_2_5
|
||||
assert limits_for(MODEL_2_0) is SEEDANCE_2_0
|
||||
|
||||
def test_matching_is_by_family_prefix_not_exact_id(self):
|
||||
"""尾部日期版本会随厂商更新而变。
|
||||
|
||||
按前缀匹配使一次模型小版本升级(如 2-5-261231)不需要改这张表。
|
||||
"""
|
||||
assert limits_for("doubao-seedance-2-5-999999") is SEEDANCE_2_5
|
||||
|
||||
def test_unknown_model_has_no_limits(self):
|
||||
assert limits_for("some-future-model") is None
|
||||
|
||||
|
||||
class TestUnknownModelsArePassedThrough:
|
||||
"""账号新开通一个模型时不该因为这张表没更新就用不了。
|
||||
|
||||
否则「开通即可用」会变成「开通还得改代码发版」。代价只是失去本地预校验。
|
||||
"""
|
||||
|
||||
def test_anything_goes_for_an_unknown_model(self):
|
||||
validate(
|
||||
"brand-new-model",
|
||||
resolution="8k",
|
||||
duration_sec=999,
|
||||
ratio="16:9",
|
||||
has_first_frame=True,
|
||||
)
|
||||
|
||||
|
||||
class TestResolutionLimits:
|
||||
def test_2_5_has_no_4k(self):
|
||||
# 官方:2.5 可选 480p / 720p / 1080p。
|
||||
with pytest.raises(ValueError, match="不支持分辨率 4k"):
|
||||
validate(MODEL_2_5, resolution="4k")
|
||||
|
||||
def test_2_0_does_have_4k(self):
|
||||
validate(MODEL_2_0, resolution="4k")
|
||||
|
||||
def test_shared_resolutions_work_on_both(self):
|
||||
for model in (MODEL_2_5, MODEL_2_0):
|
||||
for resolution in ("480p", "720p", "1080p"):
|
||||
validate(model, resolution=resolution)
|
||||
|
||||
def test_error_names_the_model_and_lists_the_options(self):
|
||||
with pytest.raises(ValueError) as exc:
|
||||
validate(MODEL_2_5, resolution="4k")
|
||||
assert MODEL_2_5 in str(exc.value)
|
||||
assert "1080p" in str(exc.value)
|
||||
|
||||
|
||||
class TestDurationLimits:
|
||||
def test_2_5_allows_up_to_30_seconds(self):
|
||||
validate(MODEL_2_5, duration_sec=30)
|
||||
|
||||
def test_2_0_caps_at_15_seconds(self):
|
||||
# 官方:2.0 系列取值范围 [4, 15]。
|
||||
validate(MODEL_2_0, duration_sec=15)
|
||||
with pytest.raises(ValueError, match=r"\[4, 15\]"):
|
||||
validate(MODEL_2_0, duration_sec=20)
|
||||
|
||||
def test_the_same_20_seconds_is_fine_on_2_5(self):
|
||||
"""同一个时长在两个模型上一个合法一个不合法——这正是需要这张表的理由。"""
|
||||
validate(MODEL_2_5, duration_sec=20)
|
||||
with pytest.raises(ValueError):
|
||||
validate(MODEL_2_0, duration_sec=20)
|
||||
|
||||
def test_below_minimum_is_rejected_on_both(self):
|
||||
for model in (MODEL_2_5, MODEL_2_0):
|
||||
with pytest.raises(ValueError):
|
||||
validate(model, duration_sec=2)
|
||||
|
||||
def test_auto_duration_bypasses_the_range_check(self):
|
||||
# -1 表示交给模型在有效范围内自选,不该被范围校验拦下。
|
||||
validate(MODEL_2_5, duration_sec=AUTO_DURATION)
|
||||
validate(MODEL_2_0, duration_sec=AUTO_DURATION)
|
||||
|
||||
def test_omitted_duration_is_not_checked(self):
|
||||
validate(MODEL_2_5)
|
||||
|
||||
|
||||
class TestRatioLockedToAdaptiveOnImageToVideo:
|
||||
"""2.5 在图生视频时输出宽高比自动与首帧图片一致,不接受指定值。"""
|
||||
|
||||
def test_2_5_rejects_a_specific_ratio_when_a_first_frame_is_given(self):
|
||||
with pytest.raises(ValueError, match="只支持 ratio=adaptive"):
|
||||
validate(MODEL_2_5, ratio="16:9", has_first_frame=True)
|
||||
|
||||
def test_2_5_accepts_adaptive_with_a_first_frame(self):
|
||||
validate(MODEL_2_5, ratio="adaptive", has_first_frame=True)
|
||||
|
||||
def test_2_5_accepts_a_specific_ratio_for_text_to_video(self):
|
||||
# 限制只在图生视频场景成立;文生视频可指定宽高比。
|
||||
validate(MODEL_2_5, ratio="16:9", has_first_frame=False)
|
||||
|
||||
def test_2_0_accepts_a_specific_ratio_even_with_a_first_frame(self):
|
||||
validate(MODEL_2_0, ratio="16:9", has_first_frame=True)
|
||||
|
||||
|
||||
class TestOutputFormatSupport:
|
||||
def test_only_2_5_supports_mov(self):
|
||||
# 我们当前不发送 output_format(默认 mp4),但这张表记录了差异,
|
||||
# 以便将来开放该参数时不必重读文档。
|
||||
assert SEEDANCE_2_5.supports_output_format is True
|
||||
assert SEEDANCE_2_0.supports_output_format is False
|
||||
@@ -382,17 +382,161 @@ class TestCoverImage:
|
||||
assert provider_with(handler).fetch("x").cover_url == "https://f.png"
|
||||
|
||||
|
||||
class TestPixelDimensionsAreLeftBlankOnPurpose:
|
||||
def test_no_invented_dimensions(self):
|
||||
"""火山只返回 ratio 与 resolution,不返回像素值。
|
||||
class TestPixelDimensions:
|
||||
"""厂商只返回 ratio 与 resolution,像素值由静态对照表换算。
|
||||
|
||||
换算需要一张随模型族变化的对照表;在 V7 目录种子定下模型族之前,
|
||||
留空比编一个可能错的尺寸好。
|
||||
"""
|
||||
不需要下载产物、不需要解码、不依赖存储方案——这是 widthPx/heightPx 在 AI
|
||||
视频产物上能零成本填真值的依据(ADR-023 §11)。
|
||||
"""
|
||||
|
||||
def test_derived_from_ratio_and_resolution(self):
|
||||
# 官方表:2.0 系列 720p + 16:9 = 1280×720。
|
||||
result = provider_with(
|
||||
lambda r: httpx.Response(200, json=OFFICIAL_SUCCESS_PAYLOAD)
|
||||
).fetch("x")
|
||||
assert result.width_px is None
|
||||
assert result.height_px is None
|
||||
# 但原始信息要透出,供上层或后续实现换算。
|
||||
assert (result.width_px, result.height_px) == (1280, 720)
|
||||
# 原始信息同时透出,便于排查与将来复核。
|
||||
assert (result.ratio, result.resolution) == ("16:9", "720p")
|
||||
|
||||
def test_the_two_cells_where_the_families_differ(self):
|
||||
"""2.5 与 2.0 的 24 格里只有两格不同:480p 的 16:9 与 9:16。
|
||||
|
||||
这两格若搞错,客户端会按错误比例渲染 480p 视频且看不出是错的。
|
||||
"""
|
||||
|
||||
def payload(model: str) -> dict:
|
||||
return {
|
||||
"id": "x",
|
||||
"model": model,
|
||||
"status": "succeeded",
|
||||
"resolution": "480p",
|
||||
"ratio": "16:9",
|
||||
"content": {"video_url": "https://v"},
|
||||
}
|
||||
|
||||
result_2_5 = provider_with(
|
||||
lambda r: httpx.Response(200, json=payload("doubao-seedance-2-5-260628"))
|
||||
).fetch("x")
|
||||
result_2_0 = provider_with(
|
||||
lambda r: httpx.Response(200, json=payload("doubao-seedance-2-0-260128"))
|
||||
).fetch("x")
|
||||
assert (result_2_5.width_px, result_2_5.height_px) == (854, 480)
|
||||
assert (result_2_0.width_px, result_2_0.height_px) == (864, 496)
|
||||
|
||||
def test_4k_resolves_only_on_2_0(self):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "x",
|
||||
"model": "doubao-seedance-2-0-260128",
|
||||
"status": "succeeded",
|
||||
"resolution": "4k",
|
||||
"ratio": "16:9",
|
||||
"content": {"video_url": "https://v"},
|
||||
},
|
||||
)
|
||||
|
||||
result = provider_with(handler).fetch("x")
|
||||
assert (result.width_px, result.height_px) == (3840, 2160)
|
||||
|
||||
def test_adaptive_ratio_yields_none_rather_than_a_guess(self):
|
||||
"""`adaptive` 时厂商自动适配首帧比例,本表无从查起。
|
||||
|
||||
留空而不猜:一个错的尺寸会让客户端按错误比例渲染,而且看不出是错的。
|
||||
"""
|
||||
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "x",
|
||||
"model": "doubao-seedance-2-5-260628",
|
||||
"status": "succeeded",
|
||||
"resolution": "720p",
|
||||
"ratio": "adaptive",
|
||||
"content": {"video_url": "https://v"},
|
||||
},
|
||||
)
|
||||
|
||||
result = provider_with(handler).fetch("x")
|
||||
assert (result.width_px, result.height_px) == (None, None)
|
||||
# 但原始值仍透出,上层至少知道厂商说了什么。
|
||||
assert result.ratio == "adaptive"
|
||||
|
||||
def test_unknown_model_family_yields_none(self):
|
||||
def handler(request):
|
||||
return httpx.Response(
|
||||
200,
|
||||
json={
|
||||
"id": "x",
|
||||
"model": "some-future-model",
|
||||
"status": "succeeded",
|
||||
"resolution": "720p",
|
||||
"ratio": "16:9",
|
||||
"content": {"video_url": "https://v"},
|
||||
},
|
||||
)
|
||||
|
||||
result = provider_with(handler).fetch("x")
|
||||
assert (result.width_px, result.height_px) == (None, None)
|
||||
|
||||
|
||||
class TestPerModelLimitsAreEnforcedBeforeCallingTheVendor:
|
||||
"""超范围的请求本地就拦下,不花一次往返去换一个厂商 400。
|
||||
|
||||
两个已接入模型的限制不同,同一组参数在一个上合法、在另一个上不合法——
|
||||
这正是需要本地预校验的理由。
|
||||
"""
|
||||
|
||||
def test_duration_beyond_the_2_0_cap_is_rejected_locally(self):
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求:本地预校验就该拦下")
|
||||
|
||||
with pytest.raises(ValueError, match=r"\[4, 15\]"):
|
||||
provider_with(handler).submit(
|
||||
video_request(model="doubao-seedance-2-0-260128", duration_sec=20)
|
||||
)
|
||||
|
||||
def test_the_same_duration_passes_on_2_5(self):
|
||||
def handler(request):
|
||||
return httpx.Response(200, json={"id": "cgt-1"})
|
||||
|
||||
assert (
|
||||
provider_with(handler).submit(
|
||||
video_request(model="doubao-seedance-2-5-260628", duration_sec=20)
|
||||
)
|
||||
== "cgt-1"
|
||||
)
|
||||
|
||||
def test_4k_is_rejected_on_2_5_but_allowed_on_2_0(self):
|
||||
def rejecting(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
with pytest.raises(ValueError, match="不支持分辨率 4k"):
|
||||
provider_with(rejecting).submit(
|
||||
video_request(model="doubao-seedance-2-5-260628", resolution="4k")
|
||||
)
|
||||
|
||||
def accepting(request):
|
||||
return httpx.Response(200, json={"id": "cgt-2"})
|
||||
|
||||
assert (
|
||||
provider_with(accepting).submit(
|
||||
video_request(model="doubao-seedance-2-0-260128", resolution="4k")
|
||||
)
|
||||
== "cgt-2"
|
||||
)
|
||||
|
||||
def test_2_5_image_to_video_rejects_a_specific_ratio(self):
|
||||
def handler(request):
|
||||
raise AssertionError("不应发出请求")
|
||||
|
||||
with pytest.raises(ValueError, match="只支持 ratio=adaptive"):
|
||||
provider_with(handler).submit(
|
||||
video_request(
|
||||
model="doubao-seedance-2-5-260628",
|
||||
input_url="https://x/a.png",
|
||||
ratio="16:9",
|
||||
)
|
||||
)
|
||||
|
||||
Reference in New Issue
Block a user