SHA256
Compare commits
5
Commits
| Author | SHA256 | Date | |
|---|---|---|---|
 曾志威
|
51bbefc6dc | ||
|
曾志威
|
cb62853e9e | ||
|
曾志威
|
702e586be6 | ||
|
曾志威
|
2a7caede7f | ||
|
曾志威
|
28bf1e4a47 |
@@ -1,306 +1,149 @@
|
||||
# ashare-data
|
||||
|
||||
A股数据抓取工具，股票列表、交易日历使用 AKShare，日线行情使用 BaoStock，其余部分按功能分别使用通达信本地缓存或 AKShare，保存到 MySQL 数据库。
|
||||
A 股数据抓取工具，正式 CLI 入口是 `python -m src.main`。支持股票列表、交易日历、日 K、5 分钟 K 抓取，并写入 MySQL。
|
||||
|
||||
## 功能概览
|
||||
## 当前正式命令
|
||||
|
||||
| 数据类型 | 说明 | 数据来源 |
|
||||
|---------|------|---------|
|
||||
| 股票列表 | 沪深A股代码、名称、上市日期 | AKShare |
|
||||
| 交易日历 | 1990年至今的交易日列表 | AKShare |
|
||||
| 日线行情 | 开高低收、成交量/额、振幅、涨跌幅、换手率（前复权，限沪深 A 股） | BaoStock 日线接口 |
|
||||
| 新浪1分钟数据 | 最近 1 分钟K线（开高低收、成交量/额） | AKShare 新浪接口 |
|
||||
| 指数日线 | 上证/沪深300/中证500/中证1000/科创50/深证成指/创业板指/中小板指 | 通达信本地缓存 |
|
||||
| 涨跌停统计 | 每日主板（10%）/ 科创创业板（20%）涨跌停数量 | stock_daily 汇总 |
|
||||
| 概念板块 | 通达信本地概念板块缓存 | 通达信本地缓存 |
|
||||
|
||||
**初始化说明**：首次运行 `--stock-info` / `--trading-day` 时会自动从 AKShare 拉取并写入数据库；`--daily` 现在改为 BaoStock 日线接口，只写 `stock_daily`；`--daily-no-data-only` 仅更新 `stock_no_data`；`--sector` / `--index` / `--intraday` 仍按通达信本地缓存工作，不再依赖 `hsjday.zip` 这类旧的日线初始化包。
|
||||
| 命令 | 作用 | 主要写入表 |
|
||||
|------|------|------------|
|
||||
| `--stock-info` | 同步股票列表 | `stock_info` |
|
||||
| `--trading-day` | 用指数日线推导交易日历 | `trading_day` |
|
||||
| `--daily` | 抓取日线行情 | `stock_daily` |
|
||||
| `--index-daily` | 增量更新指数行情页的常用指数日线 | `index_daily` |
|
||||
| `--backfill` | 补全股票日 K、5 分钟 K 及指数行情 | `stock_daily`、`stock_min5`、`index_daily`、`index_min5` |
|
||||
| `--minute` | 抓取 5 分钟 K 线 | `stock_min5`、`index_min5` |
|
||||
| `--trade-status` / `--mark-suspended` | 归档交易状态并刷新停牌标记 | `stock_trade_status` |
|
||||
| `--industry-tree` | 同步自动产业链目录与股票产业链归属 | `industry_node`、`stock_industry` |
|
||||
|
||||
## 快速开始
|
||||
|
||||
### 1. 环境要求
|
||||
### 环境要求
|
||||
|
||||
- Python >= 3.10
|
||||
- MySQL >= 8.0
|
||||
- Python `>= 3.10`
|
||||
- MySQL `>= 8.0`
|
||||
|
||||
### 2. 安装依赖
|
||||
### 安装依赖
|
||||
|
||||
```bash
|
||||
pip install pymysql sqlalchemy pyyaml pandas requests baostock akshare
|
||||
pip install -e .
|
||||
```
|
||||
|
||||
> 说明：当前只需要 `pymysql`、`sqlalchemy`、`pyyaml`、`pandas`、`requests`、`baostock` 等基础依赖。
|
||||
> 如需使用 `--sina-min1` 命令，还需要 `akshare`。
|
||||
> 通达信分钟缓存刷新已自动按最多 100 只股票一批拆分。注意通达信客户端原生只支持刷新 `1m/5m` 本地缓存。
|
||||
### 配置
|
||||
|
||||
> 仅 `--sector` / `--index` / `--intraday` 等命令需要通达信本地目录，默认读取 `config.yaml` 里的 `tdx.dir`。
|
||||
`config.yaml` 包含数据库密码，请保持在 `.gitignore` 中。
|
||||
|
||||
### 3. 配置数据库
|
||||
PowerShell 下复制模板：
|
||||
|
||||
> ⚠️ **安全提示**：`config.yaml` 包含数据库密码，**必须**保留在 `.gitignore` 中（本仓库已默认忽略）。请勿在提交前删除此规则。如曾不慎提交真实密码，参见 [TODO.md](./TODO.md) "清理 git 历史中的明文密码"。
|
||||
|
||||
复制配置文件并修改 MySQL 连接信息：
|
||||
|
||||
```bash
|
||||
cp config.example.yaml config.yaml
|
||||
```powershell
|
||||
Copy-Item config.example.yaml config.yaml
|
||||
```
|
||||
|
||||
编辑 `config.yaml`：
|
||||
|
||||
```yaml
|
||||
mysql:
|
||||
host: "localhost"
|
||||
port: 3306
|
||||
user: "root"
|
||||
password: "your_password"
|
||||
database: "ashare"
|
||||
charset: "utf8mb4"
|
||||
|
||||
fetch:
|
||||
delay: 0.1 # 请求间隔(秒)
|
||||
retry: 2 # 失败重试次数
|
||||
workers: 1 # 并发线程数
|
||||
|
||||
tdx:
|
||||
dir: "C:\\new_tdx64" # 通达信安装目录
|
||||
```
|
||||
|
||||
确保 MySQL 中已创建对应数据库：
|
||||
数据库需提前创建：
|
||||
|
||||
```sql
|
||||
CREATE DATABASE ashare CHARACTER SET utf8mb4 COLLATE utf8mb4_unicode_ci;
|
||||
```
|
||||
|
||||
### 4. 运行
|
||||
## 常用命令
|
||||
|
||||
```bash
|
||||
# 1. 先抓取股票列表（数据来源: AKShare）
|
||||
# 股票列表 / 交易日历
|
||||
python -m src.main --stock-info
|
||||
python -m src.main --trading-day --start-date 20150101 --end-date 20261231
|
||||
|
||||
# 2. 抓取交易日历（数据来源: AKShare）
|
||||
python -m src.main --trading-day --start-date 20100101 --end-date 20261231
|
||||
|
||||
# 3. 抓取全历史日线行情（数据来源: BaoStock 日线接口，限沪深 A 股）
|
||||
python -m src.main --daily --start-date 20260521 --end-date 20260521
|
||||
|
||||
# 4. 日常增量更新日线（数据来源: BaoStock 日线接口，限沪深 A 股；只写 stock_daily）
|
||||
# 日 K，默认近 30 天
|
||||
python -m src.main --daily
|
||||
python -m src.main --daily --start-date 20260616 --end-date 20260616
|
||||
python -m src.main --daily --symbol 600498,000001 --start-date 20260501 --end-date 20260517
|
||||
|
||||
# 4.1 仅更新停牌/无数据记录，不写 stock_daily（数据来源: BaoStock 日线接口，限沪深 A 股）
|
||||
python -m src.main --daily-no-data-only --start-date 20100101 --end-date 20260517
|
||||
# 指数行情（默认按各指数数据库最新日期增量更新）
|
||||
python -m src.main --index-daily
|
||||
|
||||
# 5. 下载新浪 1 分钟数据并写入 stock_min1（数据来源: AKShare 新浪接口，默认 2010 至今）
|
||||
python -m src.main --sina-min1
|
||||
# 自动补全股票日 K、分钟 K 和指数行情；首次无数据时全量，后续自动增量
|
||||
python -m src.main --backfill
|
||||
python -m src.main --backfill --start-date 20150101
|
||||
|
||||
# 6. 下载新浪 1 分钟数据并指定区间（数据来源: AKShare 新浪接口）
|
||||
python -m src.main --sina-min1 --start-date 20260501 --end-date 20260517 --symbol 600519
|
||||
# 只补最新增量窗口，默认回看 30 天
|
||||
python -m src.main --backfill --incremental --backfill-lookback-days 30
|
||||
|
||||
# 7. 抓取概念板块及成分股（数据来源: 通达信本地缓存）
|
||||
python -m src.main --sector
|
||||
# 强制扫描历史缺口
|
||||
python -m src.main --backfill --full-backfill --start-date 20150101
|
||||
|
||||
# 8. 汇总每日涨跌停统计（数据来源: stock_daily 汇总）
|
||||
python -m src.main --market-daily
|
||||
# 持续后台扫描；每轮结束后默认等待 1 小时，可用 --interval 调整
|
||||
python -m src.main --backfill --daemon --interval 3600
|
||||
|
||||
# PowerShell：将守护进程放入后台运行（当前目录应为 ashare-data）
|
||||
Start-Process python -ArgumentList '-m','src.main','--backfill','--daemon','--interval','3600' -WorkingDirectory (Get-Location) -WindowStyle Hidden
|
||||
|
||||
# 5 分钟 K
|
||||
python -m src.main --minute
|
||||
python -m src.main --minute --symbol 600519
|
||||
python -m src.main --minute --start-date 20260101 --end-date 20260618
|
||||
|
||||
# 刷新交易状态和停牌标记；需先有股票列表、交易日历、日 K
|
||||
python -m src.main --trade-status --start-date 20260601 --end-date 20260618
|
||||
|
||||
# 研报产业链树；同花顺 F10 主营和产品信息用于股票归属
|
||||
python -m src.main --industry-tree
|
||||
python -m src.main --industry-tree --industry-taxonomy-only
|
||||
python -m src.main --industry-tree --symbol 688981 --industry-refresh
|
||||
python -m src.main --industry-tree --limit 100
|
||||
```
|
||||
|
||||
### 5. 命令行参数说明
|
||||
## 默认行为
|
||||
|
||||
```
|
||||
数据抓取选项:
|
||||
--stock-info 抓取A股股票列表（数据来源: AKShare）
|
||||
--trading-day 抓取交易日历（数据来源: AKShare）
|
||||
--daily 抓取日线行情（数据来源: BaoStock 日线接口；限沪深 A 股；增量，自动分析数据缺口，已完整自动跳过；只写入 stock_daily，不更新 stock_no_data）
|
||||
--daily-no-data-only 仅更新 stock_no_data（数据来源: BaoStock 日线接口；限沪深 A 股；不写入 stock_daily）
|
||||
--sina-min1 抓取新浪 1 分钟数据并写入 stock_min1（数据来源: AKShare 新浪接口；默认全市场、默认 2010 至今，可配合 --symbol / --start-date / --end-date）
|
||||
--sector 抓取概念板块及成分股（数据来源: 通达信本地缓存；增量，本地板块已齐全时跳过）
|
||||
--market-daily 汇总每日涨跌停统计（数据来源: stock_daily 聚合）
|
||||
--symbol 指定单只或多个股票代码，逗号分隔；不传则抓取全市场，供 --sina-min1 使用
|
||||
- `--stock-info`：无日期参数，直接同步股票列表。
|
||||
- `--trading-day`：默认区间是 `19901219 ~ today`。
|
||||
- `--daily`：默认近 30 天；只写 `stock_daily`；默认跳过 `stock_trade_status` 已标记为停牌的日期，可配合 `--limit` 小批量补数。
|
||||
- `--backfill`：按交易日扫描股票日 K、分钟 K 和指数行情缺口；默认 `auto` 模式首次全量、后续增量。增量模式按数据库最新日期向前回看 30 天，日 K 和分钟 K 分别计算起点；每轮依次补股票日 K、分钟 K（其中包含指数 5 分钟行情）和指数日线。
|
||||
- `--incremental`：只扫描最新增量窗口。历史老缺口请使用 `--full-backfill`，避免增量模式遗漏窗口之外的缺口。
|
||||
- `--backfill-lookback-days`：增量回看窗口，默认 30 天；数据源有延迟或经常修订时可调大。
|
||||
- `--backfill-mode auto|incremental|full`：补全模式的完整写法。
|
||||
- `--backfill --daemon`：补全一轮后按间隔持续运行。Windows 下可用 PowerShell `Start-Process` 将该命令放入后台；日志仍写入 `logs/ashare.log`。
|
||||
- `--minute`：默认 `20150101 ~ today`，只抓 5 分钟 K；已有完整交易日和已标记停牌日会跳过。
|
||||
- `--trade-status` / `--mark-suspended`：默认近 30 天；将无日 K 的已上市股票标记为 `suspended`。全市场日 K 覆盖率低于配置阈值时改标记为 `source_error`，避免把数据异常误判成停牌。
|
||||
- `--include-suspended`：日 K、分钟 K 和 `--backfill` 默认会跳过已标记停牌日；添加该参数可忽略标记并强制重试。
|
||||
- `--symbol`：支持 `--daily`、`--minute`、`--industry-tree`。
|
||||
- `--industry-tree`：同步研报覆盖的产业链，并使用同花顺 F10 主营与产品证据将股票挂到叶子节点。当前包括半导体、机器人、商业航天、CPO、固态电池、煤化工、基础设施、低空经济、AIDC、新型储能、光伏、风电、氢能、创新药、医疗器械、新能源汽车、军工电子、消费电子、软件云与网络安全、石化与精细化工、化学制药、中药、医疗服务、医药流通、食品饮料、旅游酒店、零售商业、种植业、畜禽养殖、水产渔业、宠物、家电、纺织服装、轻工家居、教育、人力资源服务、传媒内容、交通物流、公用事业、环保、房地产、建筑材料、钢铁、有色金属和金融服务，共 45 棵树。同一股票可以进入多棵产业链；没有明确业务证据的候选保留为未匹配，不会强制挂载。概念名称和经营范围本身不参与匹配，概念解析只有通过业务上下文审核后才用于细分。共享 F10 缓存会自动发现因板块分页遗漏但业务说明明确的跨链股票；`--industry-refresh` 会补抓并更新 F10 主营介绍。
|
||||
- `python -m scripts.refresh_industry_f10`：刷新新增产业链候选的 F10 业务解析缓存，规则调整后可用于清理旧格式证据。
|
||||
|
||||
日期过滤（对日线行情、交易日历、新浪1分钟生效；新浪1分钟默认 2010 至今）:
|
||||
--start-date 开始日期，格式 YYYYMMDD
|
||||
--end-date 结束日期，格式 YYYYMMDD
|
||||
```
|
||||
### 同一来源的多个归属
|
||||
|
||||
## 数据库表结构
|
||||
`stock_industry` 按 `(source, stock_code, industry_code)` 唯一。同一股票在同一产业链内可以同时归属多个产品节点，例如刻蚀设备和薄膜沉积设备；命中细分节点后，不再重复保存其父节点。同步统计中的匹配数仍按股票计数。
|
||||
|
||||
### stock_info — 股票基本信息
|
||||
成功分类后，同步会替换本次股票在该来源下的有效归属集合，将失效节点和未匹配占位标为 `active = 0`，保留原记录；再次命中时恢复同一记录。抓取失败、未参与本次处理的股票和其他来源的归属保持不变。写入按来源使用 MySQL 连接锁串行化，每批最多处理 100 只股票，在短事务内按已查询的主键更新；新增记录通过共享会话的 `batch_upsert` 写入，失败时回滚该批。网络抓取在写入前完成。
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) PK | 股票代码 |
|
||||
| name | VARCHAR(50) | 股票名称 |
|
||||
| ipo_date | DATE | 上市日期 |
|
||||
已有数据库需要先停止产业链同步任务，在应用数据库中依次执行 [唯一索引迁移](scripts/migrations/20260925_stock_industry_multiple.sql) 和 [归属有效标记迁移](scripts/migrations/20260925_stock_industry_active.sql)，然后部署新版数据服务和展示服务。两个脚本各执行一次；若已完成唯一索引迁移，只执行有效标记迁移。新建数据库直接采用新表结构；启动时建表不会迁移已有字段或索引。
|
||||
|
||||
### stock_daily — 日线行情（前复权）
|
||||
迁移仅调整索引和增加有效标记，不推断或复制历史归属。之后运行 `python -m src.main --industry-tree` 可用有效缓存重新分类；缓存过期时会抓取 F10。需要强制重新获取业务证据时加 `--industry-refresh`。旧的“刻蚀与薄膜沉积”节点继续承接尚未成功重分的股票，避免仅同步目录或抓取失败时丢失展示；新分类只写入拆开的产品节点，旧节点没有有效归属后会从行业树隐藏。当前证据来源仍为同花顺 F10。
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 股票代码 |
|
||||
| date | DATE | 交易日期 |
|
||||
| open | FLOAT NOT NULL | 开盘价 |
|
||||
| close | FLOAT NOT NULL | 收盘价 |
|
||||
| high | FLOAT NOT NULL | 最高价 |
|
||||
| low | FLOAT NOT NULL | 最低价 |
|
||||
| volume | FLOAT NOT NULL | 成交量 |
|
||||
| turnover | FLOAT NOT NULL | 成交额 |
|
||||
| amplitude | FLOAT NOT NULL | 振幅% |
|
||||
| pct_change | FLOAT NOT NULL | 涨跌幅% |
|
||||
| change | FLOAT NOT NULL | 涨跌额 |
|
||||
| turnover_rate | FLOAT NOT NULL | 换手率% |
|
||||
## 当前主要表
|
||||
|
||||
联合主键：`(code, date)`
|
||||
|
||||
### trading_day — 交易日历（AKShare）
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| date | DATE UQ | 交易日期 |
|
||||
|
||||
### stock_no_data — 停牌/无数据记录
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 股票代码 |
|
||||
| date | DATE | 停牌/无数据日期 |
|
||||
|
||||
联合主键：`(code, date)`
|
||||
|
||||
### stock_min5 — 分钟K线
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 股票代码 |
|
||||
| datetime | DATETIME | 时间 |
|
||||
| open | FLOAT | 开盘价 |
|
||||
| high | FLOAT | 最高价 |
|
||||
| low | FLOAT | 最低价 |
|
||||
| close | FLOAT | 收盘价 |
|
||||
| volume | FLOAT | 成交量 |
|
||||
| amount | FLOAT | 成交额 |
|
||||
|
||||
联合主键：`(code, datetime)`
|
||||
|
||||
### stock_min1 — 1分钟K线
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 股票代码 |
|
||||
| datetime | DATETIME | 时间 |
|
||||
| open | FLOAT | 开盘价 |
|
||||
| high | FLOAT | 最高价 |
|
||||
| low | FLOAT | 最低价 |
|
||||
| close | FLOAT | 收盘价 |
|
||||
| volume | FLOAT | 成交量 |
|
||||
| amount | FLOAT | 成交额 |
|
||||
|
||||
联合主键：`(code, datetime)`
|
||||
|
||||
### stock_concept — 概念板块及成分股
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 股票代码 |
|
||||
| concept_code | VARCHAR(20) | 概念板块代码 |
|
||||
| concept_name | VARCHAR(100) | 概念板块名称 |
|
||||
|
||||
联合主键：`(code, concept_code)`
|
||||
|
||||
### index_daily — 指数日线行情
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| code | VARCHAR(10) | 指数代码 |
|
||||
| date | DATE | 交易日期 |
|
||||
| open | FLOAT | 开盘价 |
|
||||
| high | FLOAT | 最高价 |
|
||||
| low | FLOAT | 最低价 |
|
||||
| close | FLOAT | 收盘价 |
|
||||
| volume | FLOAT | 成交量 |
|
||||
| amount | FLOAT | 成交额 |
|
||||
| pct_change | FLOAT | 涨跌幅% |
|
||||
|
||||
联合主键：`(code, date)`
|
||||
|
||||
### market_daily — 每日涨跌停统计
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
|------|------|------|
|
||||
| date | DATE PK | 交易日期 |
|
||||
| limit_up_10 | INT | 涨停数（10% 主板） |
|
||||
| limit_up_20 | INT | 涨停数（20% 科创板/创业板） |
|
||||
| limit_down_10 | INT | 跌停数（10% 主板） |
|
||||
| limit_down_20 | INT | 跌停数（20% 科创板/创业板） |
|
||||
|
||||
> 阈值：涨停 `pct_change >= 9.8`（主板）或 `>= 19.5`（20% 板块）；跌停反向。当前可通过 `python -m src.main --market-daily` 运行 `src/fetchers/market_daily.py` 提供的 `fetch_market_daily()`。
|
||||
| 表名 | 说明 |
|
||||
|------|------|
|
||||
| `stock_info` | 股票代码、名称、上市日期 |
|
||||
| `trading_day` | 交易日历 |
|
||||
| `stock_daily` | 日线行情 |
|
||||
| `stock_min5` | 5 分钟行情 |
|
||||
| `stock_trade_status` | 交易状态归档：`trading` / `suspended` / `not_listed` / `source_error` |
|
||||
| `industry_node` | 自动产业链单父节点树 |
|
||||
| `stock_industry` | 股票产业链归属，`active = 1` 为当前有效集合 |
|
||||
|
||||
## 项目结构
|
||||
|
||||
```
|
||||
ashare-data/
|
||||
├── config.yaml # MySQL 连接配置（.gitignore）
|
||||
├── config.example.yaml # 配置文件模板
|
||||
├── pyproject.toml # 项目元数据
|
||||
├── requirements.txt # 运行依赖
|
||||
├── README.md # 项目说明（当前文档）
|
||||
├── TODO.md # 已知问题与待办事项
|
||||
├── src/
|
||||
│ ├── __init__.py
|
||||
│ ├── config.py # 配置读取模块（YAML + 环境变量 ASHARE_CONFIG）
|
||||
│ ├── log.py # 统一 logging（控制台 + logs/ashare.log 按日滚动）
|
||||
│ ├── db.py # SQLAlchemy 模型 + 批量 upsert
|
||||
│ ├── main.py # 命令行入口
|
||||
│ └── fetchers/
|
||||
│ ├── __init__.py
|
||||
│ ├── stock_list.py # 股票列表（AKShare）
|
||||
│ ├── trading_day.py # 交易日历（AKShare）
|
||||
│ ├── daily.py # 日线行情（BaoStock）
|
||||
│ ├── index.py # 主要指数日线
|
||||
│ ├── market_daily.py # 每日涨跌停统计（从 stock_daily 聚合）
|
||||
│ ├── sina_minute.py # 新浪 1 分钟数据（AKShare）
|
||||
│ └── sector.py # 概念板块及成分股（通达信本地）
|
||||
├── tests/ # pytest 测试（66 个用例，纯函数 + mock）
|
||||
├── benchmarks/ # 并发度压测脚本（不在 CI 跑，需真实 MySQL+外网）
|
||||
└── gzl/ # 选股脚本（独立子项目，可选）
|
||||
├── Selector.py
|
||||
└── select_stock.py
|
||||
```
|
||||
- `src/app/`：CLI 参数解析、命令分发和共享启动流程。
|
||||
- `src/services/`：业务流程层，CLI 只调用这里的同步/归档/补全入口。
|
||||
- `src/fetchers/`：底层行情抓取、转换和写入实现。
|
||||
- `src/industry/`：产业链目录、分类规则与证据解析；纯逻辑与网络抓取分离。
|
||||
- `src/db/models.py`：ORM 模型和分钟周期映射。
|
||||
- `src/db/runtime.py`：数据库连接、缓存、初始化和批量写入；`src.db` 保留兼容导出。
|
||||
- `src/db/industry.py`：多归属集合替换、失效归属保留和来源锁。
|
||||
- `src/sources/`：兼容导出层，保留旧路径调用。
|
||||
|
||||
## 设计说明
|
||||
|
||||
- **数据源分工**：股票列表、交易日历使用 AKShare；日线行情使用 BaoStock；概念板块、指数日线、分钟线等仍按功能读取通达信本地缓存；新浪 1 分钟数据使用 AKShare 新浪接口。
|
||||
- **线程安全**：通达信本地客户端调用集中在 `src/fetchers/tdx_client.py`。
|
||||
- **去重写入**：所有表通过 `db.batch_upsert()` 走 MySQL `INSERT ON DUPLICATE KEY UPDATE`，重复执行不会产生重复数据。
|
||||
- **全量增量**：所有模块均支持增量更新——日线按交易日对比找缺口，概念板块在本地已齐全时跳过；新浪 1 分钟数据支持按日期过滤后落库。
|
||||
- **停牌识别**：日线行情若两端已覆盖、内部仍有缺口，则视为停牌，不再重抓。
|
||||
- **表结构初始化**：`init_db()` 只负责建表。
|
||||
- **进程内缓存**：`get_stock_codes()` / `get_ipo_dates()` 缓存全量股票代码与上市日期，避免重复扫库。
|
||||
- **日志**：所有代码通过 `from src.log import get_logger` 输出；控制台 + `logs/ashare.log`（按日滚动，保留 7 天）双输出，通过 `ASHARE_LOG_LEVEL=DEBUG` 切换级别。
|
||||
|
||||
## 运行测试
|
||||
## 测试
|
||||
|
||||
```bash
|
||||
pip install pytest # 或 pip install -e ".[dev]"
|
||||
pytest -v
|
||||
pytest -q
|
||||
```
|
||||
|
||||
- 当前 66 个用例全部通过，覆盖：
|
||||
- `daily._ak_hist_to_rows` / `daily._fill_derived_fields`
|
||||
- `daily._fetch_one_stock` BaoStock 日线读取
|
||||
- `daily.fetch_daily` BaoStock 日线抓取与落库
|
||||
- `intraday._tdx_market_data_to_rows` / `intraday.fetch_intraday` 通达信分钟抓取
|
||||
- `market_daily._is_20pct` 主板/创业板/科创板/北交所判定
|
||||
- `tdx_blocks.load_infoharbor_blocks` 通达信本地板块缓存解析
|
||||
- `sector._fetch_concept_list` / `sector._fetch_concept_stocks` 通达信本地概念板块解析
|
||||
- `sector.fetch_sector` 概念板块增量跳过逻辑
|
||||
- `src.log.get_logger` 命名空间、handler 幂等、env 控制 level
|
||||
|
||||
## 性能压测
|
||||
|
||||
```bash
|
||||
python -m benchmarks.bench_daily --codes 200 --days 30 --workers 1,2,4,8
|
||||
```
|
||||
|
||||
会依次以指定的并发档位跑一遍真实抓取并输出 speedup 表，用于确定 `fetch.workers` 的最优档位。详见 [benchmarks/README.md](./benchmarks/README.md)。
|
||||
|
||||
## 选股子项目 `gzl/`
|
||||
|
||||
`gzl/` 目录是独立的选股策略原型（依赖 `scipy` + 本地 CSV，不读 MySQL）。当前与主项目 `src/` 解耦，**不参与** `python -m src.main` 的任何流程。后续是否合并到 `src/strategies/` 见 [TODO.md](./TODO.md)。
|
||||
当前测试覆盖日线/分钟转换、服务层转发、CLI 分发、产业链分类、归属集合幂等与回滚、配置切换和数据库安全护栏。归属持久化测试使用独立 SQLite 数据库，不连接业务数据库。
|
||||
|
||||
@@ -1,48 +1,62 @@
|
||||
# TODO
|
||||
|
||||
本文件只记录当前通达信本地缓存方案下，仍然值得跟进的事项。
|
||||
只记录当前仓库仍值得继续推进、且与现有 CLI 和代码状态一致的事项。
|
||||
|
||||
---
|
||||
|
||||
## 🔴 P0 - 需用户授权的高风险动作
|
||||
## P0 - 需用户授权的高风险动作
|
||||
|
||||
### 1. 清理 git 历史中的明文密码
|
||||
|
||||
- **现状**：`config.yaml` 已加入 `.gitignore`，但历史 commit 中曾提交过明文数据库密码。
|
||||
- **影响**：仓库历史中仍可直接取到旧凭据。
|
||||
- **处理思路**：
|
||||
- 现状：`config.yaml` 已加入 `.gitignore`，但历史 commit 中曾出现过明文数据库密码。
|
||||
- 风险：旧凭据仍可能从仓库历史中被恢复。
|
||||
- 处理思路：
|
||||
1. 先轮换数据库密码。
|
||||
2. 再用 `git filter-repo` 或 BFG 清理历史。
|
||||
3. 必要时强制推送并通知协作者重新克隆。
|
||||
3. 如需强制推送，提前通知协作者重新克隆。
|
||||
|
||||
---
|
||||
|
||||
## 🟢 P2 - 后续工程
|
||||
## P1 - 当前主流程后续工作
|
||||
|
||||
### 2. 北交所数据支持
|
||||
### 2. 继续收口目录重构
|
||||
|
||||
- **现状**：当前主流程对 `920xxx` 仍然显式跳过。
|
||||
- **处理思路**：调研是否能从通达信本地缓存补齐北交所日线/分钟线，如可行，再补表结构与抓取逻辑。
|
||||
- 已完成：数据库模型已迁入 `src.db.models`，连接/缓存/批量写入迁入 `src.db.runtime`；`src.db` 保持兼容导出。CLI 共享启动流程已收口到 `src.app.bootstrap`。
|
||||
- 后续：逐步把日线与分钟抓取器中可复用的编排逻辑上移到 service 层，保留兼容壳，避免一次性重写数据抓取链路。
|
||||
|
||||
### 3. 数据质量校验日报
|
||||
### 3. 数据质量报表
|
||||
|
||||
- **现状**：当前只有抓取过程日志，没有统一的每日质量报表。
|
||||
- **处理思路**：增加缺口统计、最新日期、异常波动、停牌覆盖率等摘要，便于盘后检查。
|
||||
- 现状：当前只有抓取过程日志，没有统一的盘后质量摘要。
|
||||
- 处理思路：增加最新交易日、缺口股票数、`stock_no_data` 标记数、`stock_daily` / `stock_min5` 覆盖率等汇总输出。
|
||||
|
||||
### 4. 抓取任务调度
|
||||
### 4. 后台自动调度
|
||||
|
||||
- **现状**：目前仍依赖手动执行 CLI。
|
||||
- **处理思路**：接入 APScheduler 或系统定时任务，定时跑 `--daily`、`--intraday`、`--tdx-verify-cache`。
|
||||
- 已完成：`python -m src.main --backfill` 扫描并补全日 K + 分钟 K 历史缺口；追加 `--daemon` 后按间隔持续运行。
|
||||
- 后续可选：根据部署环境补充 Windows 任务计划或 Linux systemd 的正式服务文件。
|
||||
|
||||
### 5. 质量与工程护栏
|
||||
### 5. 北交所支持评估
|
||||
|
||||
- **现状**：有 pytest，但还没有 CI 和迁移体系。
|
||||
- **处理思路**：补 GitHub Actions、Alembic、ruff / pre-commit 等基础工程化能力。
|
||||
- 现状：当前日 K 主流程仍显式跳过 `920xxx`。
|
||||
- 处理思路：单独评估北交所在现有数据源链路下的可行性，再决定是否接入。
|
||||
|
||||
---
|
||||
|
||||
## 📌 维护建议
|
||||
## P2 - 工程护栏
|
||||
|
||||
- 新增 TDX 相关抓取能力时，优先先写单测，再更新 README。
|
||||
- 若要调整本地缓存初始化方式，优先改 `src/fetchers/tdx_client.py`，避免在各个 fetcher 中重复处理下载逻辑。
|
||||
- 所有对外说明都以“通达信本地缓存”为唯一数据源口径。
|
||||
### 6. CI 与静态检查
|
||||
|
||||
- 现状：已有 pytest，但还没有 CI、lint 和提交前检查。
|
||||
- 处理思路：补 GitHub Actions、ruff、pre-commit 等基础工程化能力。
|
||||
|
||||
### 7. README / CLI 一致性测试
|
||||
|
||||
- 现状：命令和文档主要靠人工同步，容易再次写回过期描述。
|
||||
- 处理思路：增加针对 README 命令清单、数据来源标注和 `src.main` 参数的一致性测试。
|
||||
|
||||
---
|
||||
|
||||
## 维护建议
|
||||
|
||||
- 改动正式 CLI 时，优先同步 `README.md`、`tests/README.md`、`TODO.md`、`config.example.yaml`。
|
||||
- 新增抓取逻辑前，优先补测试，再补文档。
|
||||
- 继续保持数据库安全边界：不在业务代码里引入 `DELETE`、`DROP`、`ALTER TABLE`、`ADD COLUMN`。
|
||||
|
||||
+6
-28
@@ -1,11 +1,10 @@
|
||||
# benchmarks 目录
|
||||
|
||||
存放性能压测脚本。**不参与单元测试，不在 CI 跑**，需要真实 MySQL 和外网。
|
||||
存放性能压测脚本。**不参与单元测试，不在 CI 跑**，需要真实 MySQL 和可访问的行情服务器。
|
||||
|
||||
## 目的
|
||||
|
||||
验证 `fetch.workers` 在 BaoStock 全局锁 + 新浪/腾讯/东方财富 HTTP 限流并存的情况下，
|
||||
最优档位是多少。
|
||||
验证日线补数在不同小批量规模下的耗时，辅助设置 `--limit` 和抓取窗口。
|
||||
|
||||
## 使用
|
||||
|
||||
@@ -13,35 +12,14 @@
|
||||
# 先确保已抓过股票列表
|
||||
python -m src.main --stock-info
|
||||
|
||||
# 跑默认压测：200 只股票 × 近 30 天，依次跑 workers=1/2/4/8
|
||||
# 跑默认压测：200 只股票 x 近 30 天，依次跑 limit=50/100/200
|
||||
python -m benchmarks.bench_daily
|
||||
|
||||
# 自定义参数
|
||||
python -m benchmarks.bench_daily --codes 500 --days 60 --workers 1,2,4 --source baostock
|
||||
```
|
||||
|
||||
## 输出
|
||||
|
||||
每档 workers 跑完会输出：
|
||||
```
|
||||
==== workers=4 完成: 87.3s ====
|
||||
```
|
||||
|
||||
最后给出对照表与相对加速比：
|
||||
```
|
||||
workers=1 324.5s speedup=1.00x
|
||||
workers=2 178.1s speedup=1.82x
|
||||
workers=4 87.3s speedup=3.72x
|
||||
workers=8 86.9s speedup=3.74x
|
||||
python -m benchmarks.bench_daily --codes 500 --days 60 --limits 100,200,500
|
||||
```
|
||||
|
||||
## 注意
|
||||
|
||||
- 脚本会**真实命中**数据源，避免在交易时段或限流敏感时段跑；
|
||||
- 同一缺口被前一档抓走后，后续档次会出现「数据已完整」从而耗时被低估；为避免这点，
|
||||
建议每档之间用 `--days` 切到不同窗口，或预先准备一段干净缺口。
|
||||
|
||||
## 当前 TODO
|
||||
|
||||
详见 [../TODO.md](../TODO.md) P2-#4：用本脚本跑出实测数据后，把推荐 workers 写到
|
||||
`config.example.yaml` 注释里。
|
||||
- 脚本会真实命中行情服务器，避免在限流敏感时段跑；
|
||||
- 同一缺口被前一档抓走后，后续档次会出现「数据已完整」从而耗时被低估；为避免这点，建议每档之间用 `--days` 切到不同窗口，或预先准备一段干净缺口。
|
||||
|
||||
+26
-55
@@ -1,18 +1,7 @@
|
||||
"""日线行情多源轮换 + 多 worker 并发压测
|
||||
"""日线小批量补数压测。
|
||||
|
||||
用法：
|
||||
python -m benchmarks.bench_daily --codes 200 --days 30 --workers 1,2,4,8
|
||||
|
||||
行为：
|
||||
- 从 stock_info 随机/顺序取若干只股票（必须先 --stock-info）；
|
||||
- 估算它们最近 N 天的真实缺口；
|
||||
- 顺次以指定的 workers 配置跑一遍真实抓取，记录耗时、成功率、无数据数。
|
||||
- 不会清空已有数据，跑完即可被认为是常规增量。
|
||||
|
||||
说明：
|
||||
- 此脚本会真实命中外部数据源（baostock / 新浪 / 腾讯 / 东方财富），
|
||||
请在配置允许的窗口内运行，避免被限流。
|
||||
- 脚本不修改 config.yaml；要切换 workers 是通过传参，并临时覆盖 fetch.workers。
|
||||
python -m benchmarks.bench_daily --codes 200 --days 30 --limits 50,100,200
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
@@ -21,16 +10,17 @@ import argparse
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.config import load_config, get_fetch_config
|
||||
from src.db import init_db, get_stock_codes
|
||||
from src.fetchers.daily import fetch_daily
|
||||
from src.config import load_config
|
||||
from src.db import get_stock_codes, init_db
|
||||
from src.fetchers.tdx_daily import fetch_tdx_daily
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("bench")
|
||||
|
||||
|
||||
def _parse_workers(arg: str) -> list[int]:
|
||||
def _parse_limits(arg: str) -> list[int]:
|
||||
return [max(1, int(x.strip())) for x in arg.split(",") if x.strip()]
|
||||
|
||||
|
||||
@@ -45,63 +35,44 @@ def _pick_codes(n: int, seed: int | None) -> list[str]:
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="日线抓取并发度压测")
|
||||
parser = argparse.ArgumentParser(description="日线抓取小批量压测")
|
||||
parser.add_argument("--codes", type=int, default=200, help="参与压测的股票数（默认 200）")
|
||||
parser.add_argument("--days", type=int, default=30, help="时间窗口天数（默认近 30 天）")
|
||||
parser.add_argument(
|
||||
"--workers", type=str, default="1,2,4,8",
|
||||
help="并发档位列表，逗号分隔（默认 1,2,4,8）",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--source", type=str, default="all",
|
||||
choices=["baostock", "sina", "tencent", "eastmoney", "all"],
|
||||
help="数据源（默认 all 多源轮换）",
|
||||
)
|
||||
parser.add_argument("--limits", type=str, default="50,100,200", help="limit 档位列表，逗号分隔")
|
||||
parser.add_argument("--seed", type=int, default=42, help="随机种子，固定后多次跑可重复")
|
||||
args = parser.parse_args()
|
||||
|
||||
load_config()
|
||||
init_db()
|
||||
|
||||
workers_list = _parse_workers(args.workers)
|
||||
limits = _parse_limits(args.limits)
|
||||
end_date = datetime.now().strftime("%Y%m%d")
|
||||
start_date = (datetime.now() - timedelta(days=args.days)).strftime("%Y%m%d")
|
||||
sample_codes = _pick_codes(args.codes, args.seed)
|
||||
|
||||
_logger.info(
|
||||
"压测计划: codes=%d days=%d source=%s workers=%s seed=%s",
|
||||
args.codes, args.days, args.source, workers_list, args.seed,
|
||||
"压测计划: codes=%d days=%d limits=%s seed=%s 窗口:%s~%s",
|
||||
args.codes,
|
||||
args.days,
|
||||
limits,
|
||||
args.seed,
|
||||
start_date,
|
||||
end_date,
|
||||
)
|
||||
_logger.info("窗口: %s ~ %s", start_date, end_date)
|
||||
|
||||
sample_codes = _pick_codes(args.codes, args.seed)
|
||||
_logger.info("已选股票样本（前10）: %s ...", sample_codes[:10])
|
||||
|
||||
cfg = get_fetch_config()
|
||||
|
||||
results = []
|
||||
for w in workers_list:
|
||||
_logger.info("==== workers=%d 开始 ====", w)
|
||||
cfg["workers"] = w
|
||||
|
||||
from src.fetchers import daily as daily_mod
|
||||
|
||||
original_get_stock_codes = daily_mod.get_stock_codes
|
||||
daily_mod.get_stock_codes = lambda: sample_codes
|
||||
try:
|
||||
for limit in limits:
|
||||
_logger.info("==== limit=%d 开始 ====", limit)
|
||||
with patch("src.fetchers.tdx_daily.get_stock_codes", return_value=sample_codes):
|
||||
t0 = time.time()
|
||||
fetch_daily(start_date=start_date, end_date=end_date, source=args.source)
|
||||
fetch_tdx_daily(start_date=start_date, end_date=end_date, limit=limit)
|
||||
elapsed = time.time() - t0
|
||||
finally:
|
||||
daily_mod.get_stock_codes = original_get_stock_codes
|
||||
|
||||
_logger.info("==== workers=%d 完成: %.1fs ====", w, elapsed)
|
||||
results.append((w, elapsed))
|
||||
_logger.info("==== limit=%d 完成: %.1fs ====", limit, elapsed)
|
||||
results.append((limit, elapsed))
|
||||
|
||||
_logger.info("==== 汇总 ====")
|
||||
base = results[0][1] if results else 0.0
|
||||
for w, sec in results:
|
||||
speedup = base / sec if sec > 0 else 0.0
|
||||
_logger.info(" workers=%d %.1fs speedup=%.2fx", w, sec, speedup)
|
||||
for limit, sec in results:
|
||||
_logger.info(" limit=%d %.1fs", limit, sec)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
+32
-8
@@ -9,14 +9,38 @@ mysql:
|
||||
|
||||
# 数据抓取配置
|
||||
fetch:
|
||||
# 每次请求间隔（秒），避免被限流
|
||||
delay: 0.5
|
||||
# 通用请求间隔（秒），避免被限流
|
||||
delay: 1.0
|
||||
# TDX 日线单只股票请求间隔（秒）；日线数据轻量，单连接下可比分钟K快一点
|
||||
tdx_daily_delay: 0.2
|
||||
# 行情服务器连接超时（秒）
|
||||
market_timeout: 8
|
||||
# 可选：指定行情服务器，格式 host:port；不填时使用内置和默认服务器
|
||||
market_hosts:
|
||||
- "119.147.212.81:7709"
|
||||
- "106.14.95.149:7709"
|
||||
# 分钟 K 最大分页数；每页 800 根，250 页约 20 万根，覆盖多年 5 分钟历史
|
||||
minute_max_pages: 250
|
||||
# 分钟 K 单只股票请求间隔（秒）；保守一点避免被行情服务器限流
|
||||
minute_delay: 0.8
|
||||
# 失败重试次数
|
||||
retry: 3
|
||||
# 并发线程数；当前主流程以本地缓存为主，通常保持 1 即可
|
||||
# 当前正式 CLI 主流程基本串行；该字段主要供压测脚本或后续扩展使用
|
||||
workers: 1
|
||||
|
||||
# 通达信本地客户端配置
|
||||
tdx:
|
||||
# 通达信安装目录
|
||||
dir: "C:\\new_tdx64"
|
||||
# 数据库批量写入分块大小，避免单条 INSERT 过大
|
||||
db_batch_size: 1000
|
||||
# 交易状态归档：全市场日K覆盖率低于该阈值时标记 source_error，避免误判停牌
|
||||
trade_status_daily_coverage_threshold: 0.8
|
||||
# 分钟批次进度日志输出间隔
|
||||
progress_every: 50
|
||||
# 日线批次进度日志输出间隔
|
||||
daily_progress_every: 100
|
||||
# 历史缺口自动补全默认起始日期；可用 --start-date 覆盖
|
||||
backfill_start_date: "20150101"
|
||||
# --backfill --daemon 默认扫描间隔（秒）；可用 --interval 覆盖
|
||||
backfill_interval_seconds: 3600
|
||||
# 增量补全向前回看天数，覆盖最近数据修订和短期缺口
|
||||
backfill_incremental_lookback_days: 30
|
||||
# 产业链同步默认跳过已分类股票，可使用 --industry-refresh 重新抓取主营信息并分类。
|
||||
# 已有 5 分钟数据达到多少根才认为当日完整；A 股全天 5 分钟 K 通常为 48 根
|
||||
min5_complete_min_bars: 48
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
# 产业链研报覆盖审计
|
||||
|
||||
更新时间：2026-07-12
|
||||
|
||||
## 数据源与方法
|
||||
|
||||
- 研报目录：东方财富公开行业研报接口 `reportapi.eastmoney.com/report/list`。
|
||||
- 当前可访问范围：2024-07-12 至 2026-07-12，共约 3.8 万篇行业研报。
|
||||
- 筛选方法：按行业代码扫描目录，优先分析标题含“产业链、供应链、深度、专题、全景、上游、下游、核心环节、拆解”的报告。
|
||||
- 树结构：只采纳材料、设备、零部件、制造、运营、服务和应用等业务环节，不采纳申万行业名、概念标签、公司名或交易属性。
|
||||
- 股票归属：同花顺板块仅提供候选池；F10 主营业务、产品类型和产品名称是强证据，概念解析只在排除“用于、参股、拟开展、公益帮扶”等弱上下文后用于细分。概念名称和经营范围不单独决定归属。
|
||||
- 多链规则：同一股票允许进入多棵产业链，每棵产业链内保留一个证据最强的业务叶子。
|
||||
|
||||
## 已覆盖主题
|
||||
|
||||
| 产业链 | 代表性研报证据 | 提取出的核心结构 | 同花顺候选池 | 当前明确归属 |
|
||||
| --- | --- | --- | --- | ---: |
|
||||
| 半导体 | 半导体设备深度、模拟芯片专题、半导体厂务产业链、先进封装专题 | 材料、设备、EDA/IP、设计、晶圆制造、封装测试、应用 | `881121` 行业 | 176 |
|
||||
| 机器人 | 2026年机器人白皮书、宇树整机及供应链深度 | 材料、减速器、电机、控制、感知、关节、灵巧手、模型数据、本体、集成、应用 | `300816` | 466 |
|
||||
| 商业航天 | 商业航天总览、SpaceX卫星BOM全景拆解、商业航天价值闭环 | 材料部件、宇航电子、电源、卫星平台/载荷/总装、火箭动力/箭体/航电、发射、地面、运营应用 | `309130` | 182 |
|
||||
| CPO光通信 | 光模块电芯片深度、光通信中期策略、COUPE先进封装专题 | 光芯片、DSP/TIA/Driver/SerDes、无源器件、光纤、模块、LPO/CPO、交换设备、数据中心网络 | `309049` | 194 |
|
||||
| 固态电池 | 固态电池装车验证、锂电设备与固态电池产业化、固态电池核心环节 | 正负极、四类固态电解质、辅材、全固态/半固态电芯、设备、PACK/BMS、应用 | `308294` | 283 |
|
||||
| 煤化工 | 煤气化设备及工程、现代煤化工产业研究 | 资源辅料、气化设备、工程服务、煤制油气、甲醇合成氨、工业气体、下游化工与化肥 | `300084` | 57 |
|
||||
| 基础设施工程 | 工程机械深度、基础设施与岩土工程资料 | 建材装备、勘察监测、地基岩土、土木施工、总承包、交通、市政水利、港航、能源基建 | `300841` | 44 |
|
||||
| 低空经济 | 2026中国低空经济白皮书、低空基础设施白皮书、中国信通院低空产业研究 | 材料部件、飞行器制造、基础设施与信息服务、运营和场景应用四板块 | `309115` | 7 |
|
||||
| AI算力与数据中心 | AIDC中期策略、AIDC供配电深度、AI产业链互联/液冷/供电报告 | 芯片服务器、存储互联、供配电、液冷、现场能源、机房建设、IDC运营、算力服务、AI应用 | `308642` | 18 |
|
||||
| 新型储能 | 2026中国新型储能深度、全球储能系统方案、海外户储专题 | 材料、PCS、BMS/EMS、锂/钠/液流/物理/储热路线、系统集成、热管理、各侧项目与运营 | `306380` | 13 |
|
||||
| 光伏 | 光伏泛半导体专题、光伏中期策略、太空光伏研究 | 硅料、设备、硅片、电池技术、组件、辅材、逆变器支架、电站EPC运营、分布式和太空光伏 | `301079` | 29 |
|
||||
| 风电 | 风电氢能中期策略、海上风电设备深度、风电板块中期策略 | 材料、叶片、传动链、电气系统、塔筒桩基海缆、陆海整机、开发施工和运维 | `300200` | 17 |
|
||||
| 氢能 | 氢燃料电池行业概览、风电氢能中期策略、SOFC专题 | 电解制氢、工业制氢、储运、加注、电堆材料、燃料电池、交通、工业脱碳、绿色燃料 | `308491` | 16 |
|
||||
| 创新药 | 中国创新药全球竞争力、CXO深度、CAR-T/ADC/多抗专题 | 科研工具、原辅料、CRO、CDMO、小分子、生物药、ADC、多抗、细胞基因治疗、临床、生产和商业化 | `308014` | 12 |
|
||||
| 医疗器械 | 高性能医疗器械年度报告、器械中期策略、手术机器人与脑机接口专题 | 材料零部件、器械CRO、IVD、影像、生命支持、手术介入、机器人、植入、家用康复、脑机、流通服务 | `301505` | 18 |
|
||||
| 新能源汽车 | 汽车行业深度：奇瑞电动智能产业链重构、乘用车电动化复盘、智能汽车系列报告 | 动力电池、电驱电控、热管理、智能驾驶、汽车电子、智能底盘、充换电、整车与后市场 | `300008`，并入汽车零部件/整车行业池 | 85 |
|
||||
| 军工电子 | 军工信息化行业资料、雷达与军用通信专题、国防信息化研究 | 军用芯片和元件、雷达、通信数据链、导航制导、电子对抗、光电、航电、舰船电子、C4ISR | `881276`、`309128` | 36 |
|
||||
| 消费电子 | 苹果产业链框架、AI终端产业链策略、AI眼镜与XR BOM拆解 | 材料、PCB/FPC、芯片显示、光学声学、电池、模组、ODM/EMS、手机PC、穿戴、XR和AI眼镜 | `881124`、`308384`、`309152` | 49 |
|
||||
| 软件云与网络安全 | 软件重构深度、云计算业务专题、信创与网络安全行业资料 | 操作系统、数据库、中间件、开发工具、IaaS/PaaS/SaaS、网络/数据/云安全、工业软件与行业应用 | `881272`、`309020`、`300419`、`300756` | 114 |
|
||||
| 石化与精细化工 | 大炼化产业链跟踪、电子特气全景、磷化工和氟化工专题 | 油气开采油服、储运、炼油、烯烃芳烃、基础化工、树脂橡胶化纤、电子化学品、助剂、农化、氟硅化工 | 油气/炼化/化学原料/制品/化纤/电子化学品行业池 | 83 |
|
||||
| 化学制药 | 原料药板块深度、复杂制剂与一致性评价专题、制剂产业链报告 | 中间体、大宗/特色原料药、辅料包材、仿制药、口服/注射/复杂制剂、专科化学药、制剂CMO | `881140` | 26 |
|
||||
| 中药 | 中国中药行业全景图、中药与药店策略、中药材功能性食品研究 | 中药材种植交易、提取物、饮片、配方颗粒、中成药、经典名方、民族药、健康产品与流通 | `881141` | 31 |
|
||||
| 医疗服务 | 中国医院创新转化报告、医疗SPD蓝皮书、专科医疗与智慧医院专题 | 医疗SPD、临床转化、医院数字化、综合/专科医院、ICL、体检、康复、养老和互联网医疗 | `881175` | 14 |
|
||||
| 医药流通 | 医药冷链供应链全景、医药商业跟踪、连锁药店框架 | 仓储冷链、追溯、批发配送、器械配送、院内SPD、DTP特药、连锁药店、医药电商和慢病服务 | `881143` | 4 |
|
||||
| 食品饮料 | 餐饮供应链专题、乳业和黄酒深度、食品饮料策略、保健食品研究 | 原料乳业、食品配料、酒饮乳品、调味、肉制品、速冻预制、烘焙零食、营养食品和渠道 | `881134`、`881133`、`881273` | 44 |
|
||||
| 旅游酒店 | 中国连锁酒店研究、景区赛道框架、免税行业深度、入境旅游研究 | 文旅资源、酒店管理、旅游交通、景区、主题公园、酒店餐饮、邮轮、旅行社、OTA、免税和会展 | `881160`、`301223`、`308657` | 14 |
|
||||
| 零售商业 | 中国零售行业研究、全球电商趋势、专业连锁和即时零售专题 | 零售仓配和数字化、百货商超、珠宝家电药店、专业连锁、综合/跨境/直播/即时电商和品牌出海 | `881158`、`881177` | 11 |
|
||||
| 种植业 | 粮食种植深度、生物育种与种业出海专题 | 玉米/水稻/其他种子、生物育种、农技服务、粮油糖、果蔬、棉花、林业、初加工和粮食贸易 | `881101`、`881103`、`308016`、`300435` | 20 |
|
||||
| 畜禽养殖 | 生猪周期深度、肉牛原奶跟踪、动物保健研究 | 畜禽育种、饲料、疫苗兽药、生猪/禽/牛羊/奶牛养殖、屠宰加工、冷链和肉类贸易 | `881102`、`881103`、`300451`、`301576`、`308537` | 34 |
|
||||
| 水产渔业 | 世界渔业和水产养殖报告、农产品加工与水产资料 | 水产苗种、饲料、动保、海淡水和特种养殖、捕捞、水产加工、冷链与贸易 | `881102`、`881103` | 5 |
|
||||
| 宠物 | 宠物行业深度、宠物药苗国产替代、宠物食品市场研究 | 食品原料、宠物疫苗药品、主粮零食用品、宠物医院、零售电商和宠物服务 | `308588`、`308537`，并用共享F10缓存补充分页遗漏 | 7 |
|
||||
| 家电 | 家电中期策略、智能显示专题、小家电出海与厨房家电研究 | 压缩机控制器、换热和面板、综合家电、厨卫、小家电、电视、清洁电器、智能家居、ODM、渠道和售后 | 白电/黑电/小家电行业池，`300814`、`300920` | 44 |
|
||||
| 纺织服装 | 纺织行业运行报告、供应链与关税专题、运动品牌和高端服饰研究 | 纤维纱线、辅料和设备、面料、非织造、印染、成衣、家纺鞋履、运动/休闲/高端/职业装品牌 | `881135`、`881136` | 87 |
|
||||
| 轻工家居 | 家具家居出海、造纸周期、金属包装和轻工产业链研究 | 木浆废纸、板材和轻工设备、文化/包装/生活纸、纸/金属包装、定制/软体家具、五金卫浴和文具 | `881139`、`881137`、`881138` | 72 |
|
||||
| 教育 | AI教育蓝皮书、数字教育、招录培训和职业教育研究 | 教材课程、教育信息化、教学设备、职教/招录/语言/素质/驾培、在线平台和综合培训 | `881178`、`301227`、`301539` | 38 |
|
||||
| 人力资源服务 | 招聘人才供需、企业招聘策略与专业服务资料 | 在线招聘、人服数字化、派遣、灵活用工、业务外包、RPO猎头、薪税福利和管理咨询 | `881179` | 3 |
|
||||
| 传媒内容 | 出版行业数智化深度、IP经济与授权研究、游戏产业/出海专题、AI视频与短剧洞察、广告供应链和电影专题 | IP版权、AIGC与制作技术、出版、影视、动漫、短剧、游戏、院线、广电、视频平台、营销、衍生品和出海 | `881164`、`881274`、`881275` | 66 |
|
||||
| 交通物流 | 物流行业深度、航运港口专题、航空机场与快递供应链研究 | 仓储冷链、物流装备、航运港口、航空机场、铁路公路、快递、货代、跨境与末端配送 | `881152`、`881148`、`881149`、`881151` | 91 |
|
||||
| 公用事业 | 电力行业中期策略、天然气产业链与城市燃气研究 | LNG储运、火水核新能源发电、热电供热、配电网、天然气管网、城市燃气、售电和综合能源 | `881145`、`881146` | 84 |
|
||||
| 环保 | 环保产业链全景、水环境治理、固废与资源循环专题 | 水气固废设备、环境监测、水气治理、垃圾与危废、环卫、土壤修复、资源循环、设施运营和节能碳服务 | `881181`、`881284` | 83 |
|
||||
| 房地产 | 房地产行业2026年中期策略、房地产开发行业年报综述、城市更新专题 | 规划代建、城市更新、住宅/商业/园区/文旅开发、物业、商业运营、租赁住房和经纪服务 | `881153` | 71 |
|
||||
| 建筑材料 | 建材建筑新材料年度策略、建筑材料年报综述、地下管网与电子布专题 | 砂石与外加剂、水泥混凝土、玻璃玻纤、陶瓷耐材、防水、管材、板材、涂料、门窗和新型建材 | `881115` | 41 |
|
||||
| 钢铁 | 钢铁革故鼎新价值拆解、钢铁行业深度、年度供需策略 | 铁矿球团、焦化冶炼、综合钢铁、长材、板材、钢管、不锈钢、特钢、深加工和流通 | `881112` | 34 |
|
||||
| 有色金属 | 铜铝供需深度、稀土产业链、锂镍专题、小金属与贵金属系列报告 | 铜铝铅锌、锂镍钴、稀土钨钼、小金属、黄金白银、钛镁、金属粉体、再生金属和综合矿业 | `881168`、`881169`、`881170`、`881267`、`881114`、`881167` | 49 |
|
||||
| 金融服务 | 银行中期策略、大投行业务链、保险资负两端策略、信托租赁与多元金融行业分析 | 商业银行、证券经纪/投行/资管/期货、寿险财险再保险、信托、租赁、保理担保和不良资产处置 | `881155`、`881157`、`881156`、`881283` | 94 |
|
||||
|
||||
## 关键边界结论
|
||||
|
||||
- 低空经济与商业航天分树：低空围绕航空器、空域和近地运营，商业航天围绕火箭、卫星和空间服务。
|
||||
- CPO与半导体分树：DSP/TIA/Driver等光模块电芯片可同时进入半导体和CPO，光模块整机不进入半导体。
|
||||
- 固态电池与新型储能分树：固态电池按电池技术路线建树；储能按系统技术、设备和项目场景建树，可多链归属。
|
||||
- AIDC与CPO分树：AIDC覆盖服务器、电力、液冷和运营；CPO只覆盖具有明确光互联业务证据的企业。
|
||||
- 创新药与医疗器械分树：药物研发制造、CXO归创新药；器械研发注册、诊断治疗设备归医疗器械。
|
||||
- 概念名称不能作为股票归属证据；F10解析中没有明确产品、服务、项目或主营描述时保持未匹配。
|
||||
|
||||
## 待继续覆盖
|
||||
- 后续行业：煤炭资源、工程机械与专用设备、电子元件、通信服务、汽车后市场等仍需继续审计。
|
||||
+2
-3
@@ -8,9 +8,8 @@ dependencies = [
|
||||
"sqlalchemy>=2.0",
|
||||
"pyyaml",
|
||||
"pandas",
|
||||
"requests",
|
||||
"baostock>=0.9.1",
|
||||
"akshare>=1.18.60",
|
||||
"akshare",
|
||||
"pytdx>=1.72",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
||||
+2
-3
@@ -2,6 +2,5 @@ pymysql
|
||||
sqlalchemy>=2.0
|
||||
pyyaml
|
||||
pandas
|
||||
requests
|
||||
baostock>=0.9.1
|
||||
akshare>=1.18.60
|
||||
akshare
|
||||
pytdx>=1.72
|
||||
|
||||
@@ -0,0 +1,7 @@
|
||||
-- Run once after 20260925_stock_industry_multiple.sql, before deploying readers/writers.
|
||||
-- Retain superseded memberships; active = 1 denotes the current set.
|
||||
ALTER TABLE stock_industry
|
||||
ADD COLUMN active TINYINT(1) NOT NULL DEFAULT 1 AFTER industry_code,
|
||||
ADD KEY ix_stock_industry_source_active (source, active, id),
|
||||
ALGORITHM=INPLACE,
|
||||
LOCK=NONE;
|
||||
@@ -0,0 +1,7 @@
|
||||
-- Run once against the existing application database before deploying the new writers.
|
||||
-- Stop industry synchronization jobs while migrating; existing memberships are preserved.
|
||||
ALTER TABLE stock_industry
|
||||
DROP INDEX uq_stock_industry_source_stock,
|
||||
ADD UNIQUE KEY uq_stock_industry_source_stock_industry (source, stock_code, industry_code),
|
||||
ALGORITHM=INPLACE,
|
||||
LOCK=NONE;
|
||||
@@ -0,0 +1,123 @@
|
||||
"""Refresh F10 business evidence for curated industry-chain candidates."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from src.db import StockF10Evidence, StockIndustry, batch_upsert, get_session
|
||||
from src.fetchers.industry_chain import F10_PROFILE_MARKER, _refresh_ths_stock_chain_evidence
|
||||
|
||||
|
||||
SOURCES = (
|
||||
"industry_chain_la",
|
||||
"industry_chain_aidc",
|
||||
"industry_chain_es",
|
||||
"industry_chain_pv",
|
||||
"industry_chain_wind",
|
||||
"industry_chain_h2",
|
||||
"industry_chain_drug",
|
||||
"industry_chain_med",
|
||||
"industry_chain_auto",
|
||||
"industry_chain_mil",
|
||||
"industry_chain_ce",
|
||||
"industry_chain_sw",
|
||||
"industry_chain_chem",
|
||||
"industry_chain_pharm",
|
||||
"industry_chain_tcm",
|
||||
"industry_chain_care",
|
||||
"industry_chain_dist",
|
||||
"industry_chain_food",
|
||||
"industry_chain_tour",
|
||||
"industry_chain_shop",
|
||||
"industry_chain_crop",
|
||||
"industry_chain_ani",
|
||||
"industry_chain_aqua",
|
||||
"industry_chain_pet",
|
||||
"industry_chain_home",
|
||||
"industry_chain_tex",
|
||||
"industry_chain_light",
|
||||
"industry_chain_edu",
|
||||
"industry_chain_hr",
|
||||
"industry_chain_media",
|
||||
"industry_chain_trans",
|
||||
"industry_chain_util",
|
||||
"industry_chain_env",
|
||||
"industry_chain_re",
|
||||
"industry_chain_mat",
|
||||
"industry_chain_steel",
|
||||
"industry_chain_metal",
|
||||
"industry_chain_fin",
|
||||
"industry_chain_ec",
|
||||
"industry_chain_comm",
|
||||
"industry_chain_af",
|
||||
)
|
||||
|
||||
|
||||
def stale_evidence(sources: tuple[str, ...] = SOURCES) -> dict[str, str]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(StockIndustry.stock_code, StockF10Evidence.evidence_text)
|
||||
.join(StockF10Evidence, StockF10Evidence.stock_code == StockIndustry.stock_code)
|
||||
.where(
|
||||
StockIndustry.source.in_(sources),
|
||||
StockIndustry.active.is_(True),
|
||||
~StockF10Evidence.evidence_text.like(f"%{F10_PROFILE_MARKER}%"),
|
||||
)
|
||||
)
|
||||
return {str(code): str(evidence_text) for code, evidence_text in rows}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def refresh(*, workers: int = 8, sources: tuple[str, ...] = SOURCES, limit: int | None = None) -> tuple[int, int]:
|
||||
evidence_by_code = stale_evidence(sources)
|
||||
codes = sorted(evidence_by_code)
|
||||
if limit is not None:
|
||||
codes = codes[:max(int(limit), 0)]
|
||||
unsupported = {code for code in codes if code.startswith("92")}
|
||||
failed = set(unsupported)
|
||||
pending: list[dict] = []
|
||||
candidates = [code for code in codes if code not in unsupported]
|
||||
with ThreadPoolExecutor(max_workers=max(1, min(workers, 8))) as executor:
|
||||
futures = {
|
||||
executor.submit(_refresh_ths_stock_chain_evidence, code, evidence_by_code[code]): code
|
||||
for code in candidates
|
||||
}
|
||||
for completed, future in enumerate(as_completed(futures), start=1):
|
||||
stock_code = futures[future]
|
||||
try:
|
||||
pending.append({
|
||||
"stock_code": stock_code,
|
||||
"evidence_text": future.result(),
|
||||
"fetched_at": datetime.now(),
|
||||
})
|
||||
except Exception as exc:
|
||||
failed.add(stock_code)
|
||||
print(f"{stock_code} failed: {exc}", flush=True)
|
||||
if len(pending) >= 50:
|
||||
batch_upsert(StockF10Evidence, pending, ["stock_code"])
|
||||
pending.clear()
|
||||
if completed % 50 == 0 or completed == len(futures):
|
||||
print(f"refreshed {completed}/{len(futures)}, failed={len(failed)}", flush=True)
|
||||
if pending:
|
||||
batch_upsert(StockF10Evidence, pending, ["stock_code"])
|
||||
return len(codes), len(failed)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
parser = argparse.ArgumentParser(description=__doc__)
|
||||
parser.add_argument("--source", action="append", choices=SOURCES, help="Only refresh the selected chain source")
|
||||
parser.add_argument("--limit", type=int, help="Maximum stale stocks to refresh in this run")
|
||||
parser.add_argument("--workers", type=int, default=8)
|
||||
args = parser.parse_args()
|
||||
total, failures = refresh(
|
||||
workers=args.workers,
|
||||
sources=tuple(args.source) if args.source else SOURCES,
|
||||
limit=args.limit,
|
||||
)
|
||||
print(f"complete: total={total}, failed={failures}")
|
||||
@@ -0,0 +1,6 @@
|
||||
"""Application entrypoint package.
|
||||
|
||||
Phase 1 keeps the new package as a thin wrapper around the current
|
||||
``src.main`` helpers so we can migrate imports without changing behavior.
|
||||
"""
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
"""CLI 启动流程的共享编排。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections.abc import Callable
|
||||
from typing import Any
|
||||
|
||||
|
||||
def execute_cli(
|
||||
*,
|
||||
build_parser: Callable[[], Any],
|
||||
selected_commands: Callable[[Any], list[str]],
|
||||
load_config: Callable[[], Any],
|
||||
init_db: Callable[[], Any],
|
||||
run_command: Callable[[str, Any], None],
|
||||
logger,
|
||||
) -> None:
|
||||
"""执行通用 CLI 生命周期，并把具体依赖留给入口模块注入。"""
|
||||
try:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args()
|
||||
commands = selected_commands(args)
|
||||
|
||||
if not commands:
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
load_config()
|
||||
init_db()
|
||||
|
||||
for command in commands:
|
||||
run_command(command, args)
|
||||
except KeyboardInterrupt:
|
||||
logger.warning("任务被用户中断")
|
||||
return
|
||||
|
||||
logger.info("全部任务完成")
|
||||
|
||||
|
||||
__all__ = ["execute_cli"]
|
||||
@@ -0,0 +1,210 @@
|
||||
"""应用层命令解析与分发。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(description="A股数据抓取工具")
|
||||
parser.add_argument("--stock-info", action="store_true", help="同步股票列表")
|
||||
parser.add_argument("--trading-day", action="store_true", help="使用指数日线推导交易日历")
|
||||
parser.add_argument("--daily", action="store_true", help="抓取日线行情并写入 stock_daily")
|
||||
parser.add_argument("--index-daily", action="store_true", help="增量更新常用指数日线行情")
|
||||
parser.add_argument(
|
||||
"--backfill",
|
||||
"--auto-backfill",
|
||||
dest="backfill",
|
||||
action="store_true",
|
||||
help="扫描并补全历史缺失的日K和分钟K",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--daemon",
|
||||
"--backfill-daemon",
|
||||
"--background",
|
||||
dest="backfill_daemon",
|
||||
action="store_true",
|
||||
help="让 --backfill 按固定间隔持续运行",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--interval",
|
||||
"--backfill-interval",
|
||||
dest="backfill_interval",
|
||||
type=int,
|
||||
help="--backfill 守护模式的扫描间隔（秒），默认 3600",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--backfill-mode",
|
||||
choices=("auto", "incremental", "full"),
|
||||
default="auto",
|
||||
help="补全模式：auto 首次全量、后续增量；incremental 只扫最新窗口；full 扫描历史起点",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--incremental",
|
||||
dest="backfill_mode",
|
||||
action="store_const",
|
||||
const="incremental",
|
||||
help="--backfill 的增量模式别名",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--full-backfill",
|
||||
"--backfill-full",
|
||||
dest="backfill_mode",
|
||||
action="store_const",
|
||||
const="full",
|
||||
help="强制从历史起始日期扫描全部缺口",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--backfill-lookback-days",
|
||||
type=int,
|
||||
help="增量模式向前回看天数，默认 30",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--trade-status",
|
||||
"--mark-suspended",
|
||||
dest="trade_status",
|
||||
action="store_true",
|
||||
help="归档交易状态并刷新停牌标记",
|
||||
)
|
||||
parser.add_argument("--industry-tree", action="store_true", help="同步自动产业链和股票当前产业链归属")
|
||||
parser.add_argument("--industry-active", action="store_true", help="批量分类情绪导图中未入产业链的活跃股票")
|
||||
parser.add_argument("--industry-taxonomy-only", action="store_true", help="仅同步产业目录，配合 --industry-tree")
|
||||
parser.add_argument("--industry-refresh", action="store_true", help="重新拉取已有股票的产业归属，配合 --industry-tree")
|
||||
parser.add_argument("--industry-delay", type=float, help="产业归属请求间隔秒数，配合 --industry-tree")
|
||||
parser.add_argument(
|
||||
"--minute",
|
||||
dest="minute",
|
||||
action="store_true",
|
||||
help="抓取 5 分钟 K 并写入 stock_min5",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--tdx-minute",
|
||||
dest="minute",
|
||||
action="store_true",
|
||||
help=argparse.SUPPRESS,
|
||||
)
|
||||
parser.add_argument("--start-date", type=str, help="开始日期 YYYYMMDD")
|
||||
parser.add_argument("--end-date", type=str, help="结束日期 YYYYMMDD")
|
||||
parser.add_argument("--symbol", type=str, help="指定单只或多个股票代码，逗号分隔；支持 --daily / --minute")
|
||||
parser.add_argument("--limit", type=int, help="限制本次抓取待补股票数，便于小批量慢速补数")
|
||||
parser.add_argument(
|
||||
"--include-suspended",
|
||||
action="store_true",
|
||||
help="抓取日K/分钟K时忽略已归档的停牌标记，强制重试",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def selected_commands(args) -> list[str]:
|
||||
commands = []
|
||||
if args.stock_info:
|
||||
commands.append("stock_info")
|
||||
if args.trading_day:
|
||||
commands.append("trading_day")
|
||||
if args.daily:
|
||||
commands.append("daily")
|
||||
if args.index_daily:
|
||||
commands.append("index_daily")
|
||||
if args.backfill or args.backfill_daemon or args.backfill_mode != "auto":
|
||||
commands.append("backfill")
|
||||
if args.trade_status:
|
||||
commands.append("trade_status")
|
||||
if args.minute:
|
||||
commands.append("minute")
|
||||
if args.industry_tree:
|
||||
commands.append("industry_tree")
|
||||
if args.industry_active:
|
||||
commands.append("industry_active")
|
||||
return commands
|
||||
|
||||
|
||||
def _default_end_date() -> str:
|
||||
return datetime.now().strftime("%Y%m%d")
|
||||
|
||||
|
||||
def run_command(command: str, args) -> None:
|
||||
if command == "stock_info":
|
||||
from src.services.reference_service import sync_stock_info
|
||||
sync_stock_info()
|
||||
return
|
||||
|
||||
if command == "trading_day":
|
||||
from src.services.reference_service import sync_trading_days
|
||||
sync_trading_days(
|
||||
args.start_date or "19901219",
|
||||
args.end_date or _default_end_date(),
|
||||
)
|
||||
return
|
||||
|
||||
if command == "daily":
|
||||
from src.services.daily_service import sync_daily
|
||||
sync_daily(
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
symbol=args.symbol,
|
||||
limit=args.limit,
|
||||
skip_suspended=not args.include_suspended,
|
||||
)
|
||||
return
|
||||
|
||||
if command == "index_daily":
|
||||
from src.services.index_service import sync_index_daily
|
||||
sync_index_daily(start_date=args.start_date, end_date=args.end_date)
|
||||
return
|
||||
|
||||
if command == "backfill":
|
||||
from src.services.backfill_service import run_backfill
|
||||
run_backfill(
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
symbol=args.symbol,
|
||||
limit=args.limit,
|
||||
daemon=args.backfill_daemon,
|
||||
interval_seconds=args.backfill_interval,
|
||||
mode=args.backfill_mode,
|
||||
lookback_days=args.backfill_lookback_days,
|
||||
skip_suspended=not args.include_suspended,
|
||||
)
|
||||
return
|
||||
|
||||
if command == "trade_status":
|
||||
from src.services.trade_status_service import archive_trade_status
|
||||
archive_trade_status(start_date=args.start_date, end_date=args.end_date)
|
||||
return
|
||||
|
||||
if command in {"minute", "tdx_minute"}:
|
||||
from src.services.minute_service import sync_minute
|
||||
sync_minute(
|
||||
symbol=args.symbol,
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
limit=args.limit,
|
||||
skip_suspended=not args.include_suspended,
|
||||
)
|
||||
return
|
||||
|
||||
if command == "industry_tree":
|
||||
from src.services.industry_service import sync_industry_tree
|
||||
sync_industry_tree(
|
||||
symbol=args.symbol,
|
||||
limit=args.limit,
|
||||
taxonomy_only=args.industry_taxonomy_only,
|
||||
refresh=args.industry_refresh,
|
||||
delay_seconds=args.industry_delay,
|
||||
)
|
||||
return
|
||||
|
||||
if command == "industry_active":
|
||||
from src.fetchers.industry_chain import classify_unclassified_active
|
||||
classify_unclassified_active(
|
||||
end_date=args.end_date,
|
||||
refresh=args.industry_refresh,
|
||||
limit=args.limit,
|
||||
)
|
||||
return
|
||||
|
||||
raise ValueError(f"未知命令: {command}")
|
||||
|
||||
|
||||
__all__ = ["build_parser", "selected_commands", "run_command"]
|
||||
@@ -0,0 +1,25 @@
|
||||
"""应用层主入口。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.app.bootstrap import execute_cli
|
||||
from src.app.commands import build_parser, run_command, selected_commands
|
||||
from src.core.config import load_config
|
||||
from src.db import init_db
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("main")
|
||||
|
||||
|
||||
def main():
|
||||
execute_cli(
|
||||
build_parser=build_parser,
|
||||
selected_commands=selected_commands,
|
||||
load_config=load_config,
|
||||
init_db=init_db,
|
||||
run_command=run_command,
|
||||
logger=_logger,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["build_parser", "selected_commands", "run_command", "main"]
|
||||
@@ -1,125 +0,0 @@
|
||||
"""BaoStock 连接管理器 - 集中管理 login/logout/线程锁/代码格式转换。
|
||||
|
||||
BaoStock 的 query_xxx() 不是线程安全的，所有查询需通过同一把锁串行化。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import threading
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
|
||||
import baostock as bs
|
||||
|
||||
from src.log import get_logger
|
||||
|
||||
_lock = threading.Lock()
|
||||
_logged_in = False
|
||||
_logger = get_logger("baostock")
|
||||
|
||||
QUERY_TIMEOUT = 60
|
||||
MAX_RETRY = 3
|
||||
|
||||
|
||||
def bs_login():
|
||||
"""全局只 login 一次。"""
|
||||
global _logged_in
|
||||
with _lock:
|
||||
if not _logged_in:
|
||||
bs.login()
|
||||
_logged_in = True
|
||||
|
||||
|
||||
def bs_logout():
|
||||
"""程序退出时调用。"""
|
||||
global _logged_in
|
||||
with _lock:
|
||||
if _logged_in:
|
||||
bs.logout()
|
||||
_logged_in = False
|
||||
|
||||
|
||||
def _relogin():
|
||||
"""断线重连（调用方需持有 _lock）。"""
|
||||
global _logged_in
|
||||
try:
|
||||
bs.logout()
|
||||
except Exception:
|
||||
pass
|
||||
_logged_in = False
|
||||
time.sleep(1)
|
||||
bs.login()
|
||||
_logged_in = True
|
||||
_logger.info("已重连")
|
||||
|
||||
|
||||
@contextmanager
|
||||
def bs_query(query_fn, *args, **kwargs):
|
||||
"""加锁执行 BaoStock 查询，yield ResultData。
|
||||
|
||||
连接断开或超时自动重连，最多重试 MAX_RETRY 次。
|
||||
"""
|
||||
bs_login()
|
||||
short_name = query_fn.__name__.replace("query_", "")
|
||||
sig_parts = [repr(a) for a in args]
|
||||
sig_parts += [f"{k}={v!r}" for k, v in kwargs.items()]
|
||||
sig = ", ".join(sig_parts)
|
||||
_logger.debug("%s(%s)", short_name, sig)
|
||||
|
||||
result_box = [None]
|
||||
exc_box = [None]
|
||||
|
||||
def _run():
|
||||
try:
|
||||
result_box[0] = query_fn(*args, **kwargs)
|
||||
except Exception as e:
|
||||
exc_box[0] = e
|
||||
|
||||
with _lock:
|
||||
last_err = None
|
||||
for attempt in range(1, MAX_RETRY + 1):
|
||||
result_box[0] = None
|
||||
exc_box[0] = None
|
||||
|
||||
t = threading.Thread(target=_run, daemon=True)
|
||||
t.start()
|
||||
t.join(timeout=QUERY_TIMEOUT)
|
||||
|
||||
if t.is_alive():
|
||||
_logger.warning("查询超时(%ss)，重连(第%d次)", QUERY_TIMEOUT, attempt)
|
||||
_relogin()
|
||||
last_err = TimeoutError(f"BaoStock query timeout: {short_name}")
|
||||
continue
|
||||
|
||||
if exc_box[0] is not None:
|
||||
err_msg = str(exc_box[0])
|
||||
if "10057" in err_msg or "连接" in err_msg or "socket" in err_msg.lower() or "接收数据" in err_msg:
|
||||
_logger.warning("连接断开，重连(第%d次)", attempt)
|
||||
_relogin()
|
||||
last_err = exc_box[0]
|
||||
continue
|
||||
raise exc_box[0]
|
||||
|
||||
rs = result_box[0]
|
||||
if hasattr(rs, "error_code") and rs.error_code != "0" and "login" in rs.error_msg.lower():
|
||||
_logger.warning("未登录，重连(第%d次)", attempt)
|
||||
_relogin()
|
||||
last_err = RuntimeError(f"BaoStock not logged in: {rs.error_msg}")
|
||||
continue
|
||||
|
||||
yield result_box[0]
|
||||
return
|
||||
|
||||
raise last_err or RuntimeError("BaoStock query failed after retries")
|
||||
|
||||
|
||||
def code_to_bs(code: str) -> str | None:
|
||||
"""纯数字代码转 BaoStock 格式：`600000` -> `sh.600000`。"""
|
||||
code = (code or "").strip()
|
||||
if not code or len(code) != 6 or not code.isdigit():
|
||||
return None
|
||||
if code.startswith("6"):
|
||||
return f"sh.{code}"
|
||||
if code.startswith(("0", "3")):
|
||||
return f"sz.{code}"
|
||||
return None
|
||||
+16
-32
@@ -3,20 +3,22 @@
|
||||
import os
|
||||
import yaml
|
||||
from pathlib import Path
|
||||
from sqlalchemy.engine import URL
|
||||
|
||||
_CONFIG = None
|
||||
_CONFIG_PATH: Path | None = None
|
||||
|
||||
|
||||
def load_config(config_path: str | None = None) -> dict:
|
||||
global _CONFIG
|
||||
if _CONFIG is not None:
|
||||
return _CONFIG
|
||||
global _CONFIG, _CONFIG_PATH
|
||||
|
||||
# 支持通过环境变量 ASHARE_CONFIG 指定配置文件路径
|
||||
if config_path is None:
|
||||
config_path = os.environ.get("ASHARE_CONFIG", "config.yaml")
|
||||
|
||||
path = Path(config_path)
|
||||
path = Path(config_path).expanduser().resolve()
|
||||
if _CONFIG is not None and _CONFIG_PATH == path:
|
||||
return _CONFIG
|
||||
if not path.exists():
|
||||
raise FileNotFoundError(
|
||||
f"配置文件不存在: {path}\n"
|
||||
@@ -24,7 +26,11 @@ def load_config(config_path: str | None = None) -> dict:
|
||||
)
|
||||
|
||||
with open(path, "r", encoding="utf-8") as f:
|
||||
_CONFIG = yaml.safe_load(f)
|
||||
config = yaml.safe_load(f)
|
||||
if not isinstance(config, dict):
|
||||
raise ValueError("配置文件必须是 YAML 对象，不能为空或列表")
|
||||
_CONFIG = config
|
||||
_CONFIG_PATH = path
|
||||
return _CONFIG
|
||||
|
||||
|
||||
@@ -33,11 +39,11 @@ def get_mysql_url(config: dict | None = None) -> str:
|
||||
if config is None:
|
||||
config = load_config()
|
||||
m = config["mysql"]
|
||||
return (
|
||||
f"mysql+pymysql://{m['user']}:{m['password']}"
|
||||
f"@{m['host']}:{m['port']}/{m['database']}"
|
||||
f"?charset={m.get('charset', 'utf8mb4')}"
|
||||
)
|
||||
return URL.create(
|
||||
"mysql+pymysql", username=str(m["user"]), password=str(m["password"]),
|
||||
host=m["host"], port=int(m["port"]), database=m["database"],
|
||||
query={"charset": m.get("charset", "utf8mb4")},
|
||||
).render_as_string(hide_password=False)
|
||||
|
||||
|
||||
def get_fetch_config(config: dict | None = None) -> dict:
|
||||
@@ -45,25 +51,3 @@ def get_fetch_config(config: dict | None = None) -> dict:
|
||||
if config is None:
|
||||
config = load_config()
|
||||
return config.get("fetch", {"delay": 0.5, "retry": 3})
|
||||
|
||||
|
||||
def get_tdx_config(config: dict | None = None) -> dict:
|
||||
"""返回通达信相关配置。"""
|
||||
if config is None:
|
||||
config = load_config()
|
||||
return config.get("tdx", {})
|
||||
|
||||
|
||||
def get_tdx_dir(config: dict | None = None) -> str | None:
|
||||
"""返回通达信安装目录。"""
|
||||
tdx_cfg = get_tdx_config(config)
|
||||
tdx_dir = tdx_cfg.get("dir")
|
||||
return str(tdx_dir) if tdx_dir else None
|
||||
|
||||
|
||||
def get_tdx_bootstrap_url(config: dict | None = None) -> str:
|
||||
"""返回通达信日线缓存初始化包地址。"""
|
||||
if config is None:
|
||||
config = load_config()
|
||||
tdx_cfg = config.get("tdx", {})
|
||||
return str(tdx_cfg.get("bootstrap_url") or "https://data.tdx.com.cn/vipdoc/hsjday.zip")
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""项目基础设施层。"""
|
||||
@@ -0,0 +1,13 @@
|
||||
"""兼容导出：后续迁移到 core 配置层。"""
|
||||
|
||||
from src.config import (
|
||||
get_fetch_config,
|
||||
get_mysql_url,
|
||||
load_config,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"load_config",
|
||||
"get_mysql_url",
|
||||
"get_fetch_config",
|
||||
]
|
||||
@@ -0,0 +1,5 @@
|
||||
"""兼容导出：后续迁移到 core 日志层。"""
|
||||
|
||||
from src.log import get_logger
|
||||
|
||||
__all__ = ["get_logger"]
|
||||
@@ -1,278 +0,0 @@
|
||||
"""数据库模型定义与连接管理
|
||||
|
||||
数据源：按功能分别使用 Baostock / AKShare / 通达信本地缓存
|
||||
表结构概览：
|
||||
- stock_info: 股票基本信息（含上市日期，用于跳过未上市股票）
|
||||
- stock_daily: 日线行情（含振幅/涨跌幅/换手率）
|
||||
- trading_day: 交易日历（用于判断数据完整性）
|
||||
- stock_no_data: 无数据/停牌记录（避免重复抓取）
|
||||
- stock_min1: 1分钟K线
|
||||
- stock_min5: 5分钟K线
|
||||
- index_daily: 主要指数日线
|
||||
- market_daily: 每日涨跌停统计（10%/20% 板块分别计数）
|
||||
- stock_sector: 行业 + 地域分类
|
||||
- stock_concept: 概念板块及成分股
|
||||
"""
|
||||
|
||||
from sqlalchemy import (
|
||||
Column, String, Date, DateTime, Float, Integer,
|
||||
UniqueConstraint, Index, create_engine, func, select, text,
|
||||
)
|
||||
from sqlalchemy.orm import DeclarativeBase, Session, sessionmaker
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
|
||||
from src.config import get_mysql_url
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("db")
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
|
||||
|
||||
# ── 股票基本信息 ──
|
||||
class StockInfo(Base):
|
||||
__tablename__ = "stock_info"
|
||||
|
||||
code = Column(String(10), primary_key=True, comment="股票代码")
|
||||
name = Column(String(50), comment="股票名称")
|
||||
ipo_date = Column(Date, comment="上市日期")
|
||||
|
||||
|
||||
# ── 日线行情 ──
|
||||
class StockDaily(Base):
|
||||
__tablename__ = "stock_daily"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_daily_code_date"),
|
||||
Index("ix_daily_date", "date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
open = Column(Float, nullable=False, comment="开盘价")
|
||||
close = Column(Float, nullable=False, comment="收盘价")
|
||||
high = Column(Float, nullable=False, comment="最高价")
|
||||
low = Column(Float, nullable=False, comment="最低价")
|
||||
volume = Column(Float, nullable=False, comment="成交量")
|
||||
turnover = Column(Float, nullable=False, comment="成交额")
|
||||
amplitude = Column(Float, nullable=False, comment="振幅%")
|
||||
pct_change = Column(Float, nullable=False, comment="涨跌幅%")
|
||||
change = Column(Float, nullable=False, comment="涨跌额")
|
||||
turnover_rate = Column(Float, nullable=False, comment="换手率%")
|
||||
|
||||
|
||||
# ── 指数日线行情 ──
|
||||
class IndexDaily(Base):
|
||||
__tablename__ = "index_daily"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_index_code_date"),
|
||||
Index("ix_index_date", "date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="指数代码")
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
pct_change = Column(Float, comment="涨跌幅%")
|
||||
|
||||
|
||||
# ── 交易日历 ──
|
||||
class TradingDay(Base):
|
||||
__tablename__ = "trading_day"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("date", name="uq_trading_day_date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
|
||||
|
||||
# ── 无数据/停牌记录（按天粒度） ──
|
||||
class StockNoData(Base):
|
||||
__tablename__ = "stock_no_data"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_nodata_code_date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
date = Column(Date, nullable=False, comment="停牌/无数据日期")
|
||||
created_at = Column(DateTime, server_default=func.now(), comment="记录时间")
|
||||
|
||||
|
||||
# ── 5分钟K线 ──
|
||||
class StockMin1(Base):
|
||||
__tablename__ = "stock_min1"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "datetime", name="uq_min1_code_dt"),
|
||||
Index("ix_min1_datetime", "datetime"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
datetime = Column(DateTime, nullable=False, comment="时间")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
|
||||
|
||||
# ── 5分钟K线 ──
|
||||
class StockMin5(Base):
|
||||
__tablename__ = "stock_min5"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "datetime", name="uq_min5_code_dt"),
|
||||
Index("ix_min5_datetime", "datetime"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
datetime = Column(DateTime, nullable=False, comment="时间")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
|
||||
|
||||
# ── 每日涨跌停统计 ──
|
||||
class MarketDaily(Base):
|
||||
__tablename__ = "market_daily"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("date", name="uq_market_daily_date"),
|
||||
)
|
||||
|
||||
date = Column(Date, primary_key=True, comment="交易日期")
|
||||
limit_up_10 = Column(Integer, comment="涨停数（10%主板）")
|
||||
limit_up_20 = Column(Integer, comment="涨停数（20%科创/创业板）")
|
||||
limit_down_10 = Column(Integer, comment="跌停数（10%主板）")
|
||||
limit_down_20 = Column(Integer, comment="跌停数（20%科创/创业板）")
|
||||
|
||||
|
||||
# ── 行业+地域分类 ──
|
||||
class StockSector(Base):
|
||||
__tablename__ = "stock_sector"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", name="uq_sector_code"),
|
||||
)
|
||||
|
||||
code = Column(String(10), primary_key=True, comment="股票代码")
|
||||
industry = Column(String(100), comment="行业分类")
|
||||
region = Column(String(20), comment="省份/地域")
|
||||
|
||||
|
||||
class StockConcept(Base):
|
||||
__tablename__ = "stock_concept"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "concept_code", name="uq_concept_code"),
|
||||
Index("ix_concept_code", "concept_code"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
concept_code = Column(String(20), nullable=False, comment="概念板块代码")
|
||||
concept_name = Column(String(100), nullable=False, comment="概念板块名称")
|
||||
|
||||
|
||||
# ── 数据库连接管理 ──
|
||||
_engine = None
|
||||
_SessionFactory = None
|
||||
_stock_codes_cache: list[str] | None = None
|
||||
_ipo_dates_cache: dict[str, str] | None = None
|
||||
|
||||
|
||||
def get_stock_codes() -> list[str]:
|
||||
"""获取全部股票代码（进程内缓存，避免重复查询）"""
|
||||
global _stock_codes_cache
|
||||
if _stock_codes_cache is None:
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(select(StockInfo.code))
|
||||
_stock_codes_cache = [row[0] for row in result]
|
||||
finally:
|
||||
session.close()
|
||||
return _stock_codes_cache
|
||||
|
||||
|
||||
def invalidate_stock_codes_cache() -> None:
|
||||
"""失效股票代码缓存，便于股票列表更新后重新读取。"""
|
||||
global _stock_codes_cache
|
||||
_stock_codes_cache = None
|
||||
|
||||
|
||||
def invalidate_ipo_dates_cache() -> None:
|
||||
"""失效上市日期缓存，便于股票列表更新后重新读取。"""
|
||||
global _ipo_dates_cache
|
||||
_ipo_dates_cache = None
|
||||
|
||||
|
||||
def get_ipo_dates() -> dict[str, str]:
|
||||
"""获取全部股票上市日期（进程内缓存，避免重复查询）"""
|
||||
global _ipo_dates_cache
|
||||
if _ipo_dates_cache is None:
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(select(StockInfo.code, StockInfo.ipo_date))
|
||||
_ipo_dates_cache = {code: str(ipo) for code, ipo in result if ipo}
|
||||
finally:
|
||||
session.close()
|
||||
return _ipo_dates_cache
|
||||
|
||||
|
||||
def get_engine():
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(get_mysql_url(), pool_size=5, pool_recycle=3600)
|
||||
return _engine
|
||||
|
||||
|
||||
def get_session() -> Session:
|
||||
global _SessionFactory
|
||||
if _SessionFactory is None:
|
||||
_SessionFactory = sessionmaker(bind=get_engine())
|
||||
return _SessionFactory()
|
||||
|
||||
|
||||
def init_db():
|
||||
engine = get_engine()
|
||||
Base.metadata.create_all(engine)
|
||||
_logger.info("数据库表初始化完成")
|
||||
|
||||
|
||||
def batch_upsert(model_cls: type[Base], rows: list[dict], index_columns: list[str]):
|
||||
"""MySQL批量upsert：INSERT ON DUPLICATE KEY UPDATE"""
|
||||
if not rows:
|
||||
return
|
||||
session = get_session()
|
||||
try:
|
||||
stmt = mysql_insert(model_cls).values(rows)
|
||||
input_keys = set(rows[0].keys())
|
||||
update_dict = {
|
||||
col.name: stmt.inserted[col.name]
|
||||
for col in model_cls.__table__.columns
|
||||
if col.name in input_keys
|
||||
and col.name not in index_columns
|
||||
and not col.primary_key
|
||||
and not col.server_default
|
||||
}
|
||||
if update_dict:
|
||||
stmt = stmt.on_duplicate_key_update(**update_dict)
|
||||
else:
|
||||
stmt = stmt.on_duplicate_key_update(**{index_columns[0]: stmt.inserted[index_columns[0]]})
|
||||
session.execute(stmt)
|
||||
session.commit()
|
||||
except Exception as e:
|
||||
session.rollback()
|
||||
raise e
|
||||
finally:
|
||||
session.close()
|
||||
@@ -0,0 +1,59 @@
|
||||
"""数据库公共 API。
|
||||
|
||||
模型定义位于 :mod:`src.db.models`，连接和写入能力位于
|
||||
:mod:`src.db.runtime`。本模块保留原有导入路径，避免调用方迁移。
|
||||
"""
|
||||
|
||||
from src.db.models import (
|
||||
Base,
|
||||
IndustryNode,
|
||||
IndexDaily,
|
||||
IndexMin5,
|
||||
STOCK_MINUTE_MODELS,
|
||||
StockDaily,
|
||||
StockF10Evidence,
|
||||
StockInfo,
|
||||
StockIndustry,
|
||||
StockMin5,
|
||||
StockSector,
|
||||
StockTradeStatus,
|
||||
TradingDay,
|
||||
get_stock_minute_model,
|
||||
normalize_minute_freq,
|
||||
)
|
||||
from src.db.runtime import (
|
||||
batch_upsert,
|
||||
get_engine,
|
||||
get_ipo_dates,
|
||||
get_session,
|
||||
get_stock_codes,
|
||||
init_db,
|
||||
invalidate_ipo_dates_cache,
|
||||
invalidate_stock_codes_cache,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"Base",
|
||||
"StockInfo",
|
||||
"StockDaily",
|
||||
"StockF10Evidence",
|
||||
"IndexDaily",
|
||||
"IndexMin5",
|
||||
"TradingDay",
|
||||
"StockTradeStatus",
|
||||
"StockMin5",
|
||||
"StockSector",
|
||||
"IndustryNode",
|
||||
"StockIndustry",
|
||||
"STOCK_MINUTE_MODELS",
|
||||
"normalize_minute_freq",
|
||||
"get_stock_minute_model",
|
||||
"get_engine",
|
||||
"get_session",
|
||||
"init_db",
|
||||
"batch_upsert",
|
||||
"get_stock_codes",
|
||||
"get_ipo_dates",
|
||||
"invalidate_stock_codes_cache",
|
||||
"invalidate_ipo_dates_cache",
|
||||
]
|
||||
@@ -0,0 +1,112 @@
|
||||
"""Replace complete membership sets for successfully classified stocks."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from collections import defaultdict
|
||||
from contextlib import contextmanager
|
||||
import hashlib
|
||||
|
||||
from sqlalchemy import select, text, update
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from src.db.models import StockIndustry
|
||||
from src.db.runtime import batch_upsert, get_engine
|
||||
|
||||
|
||||
def _replace_membership_batch(session: Session, rows: list[dict]) -> None:
|
||||
"""The caller serializes writers; each batch contains complete stock membership sets.
|
||||
|
||||
Keep inserts, primary-key updates and stale-row deactivation in the same short session
|
||||
transaction so a failed write never leaves half of a stock's memberships visible.
|
||||
No network fetching takes place in this transaction.
|
||||
"""
|
||||
if not rows:
|
||||
return
|
||||
source = rows[0]["source"]
|
||||
stock_codes = sorted({row["stock_code"] for row in rows})
|
||||
desired = {(row["stock_code"], row["industry_code"]): row for row in rows}
|
||||
existing = {
|
||||
(stock_code, industry_code): (row_id, active)
|
||||
for row_id, stock_code, industry_code, active in session.execute(
|
||||
select(StockIndustry.id, StockIndustry.stock_code, StockIndustry.industry_code, StockIndustry.active)
|
||||
.where(StockIndustry.source == source, StockIndustry.stock_code.in_(stock_codes))
|
||||
)
|
||||
}
|
||||
updates = [
|
||||
{"id": existing[key][0], "active": True, "effective_date": row.get("effective_date"), "updated_at": row["updated_at"]}
|
||||
for key, row in desired.items() if key in existing
|
||||
]
|
||||
additions = [{**row, "active": True} for key, row in desired.items() if key not in existing]
|
||||
checked_at = {row["stock_code"]: row["updated_at"] for row in rows}
|
||||
stale_updates = [
|
||||
{"id": row_id, "active": False, "updated_at": checked_at[key[0]]}
|
||||
for key, (row_id, active) in existing.items() if key not in desired and active
|
||||
]
|
||||
if updates:
|
||||
# ORM bulk UPDATE binds each previously selected primary key.
|
||||
session.execute(update(StockIndustry), updates)
|
||||
if additions:
|
||||
batch_upsert(StockIndustry, additions, ["source", "stock_code", "industry_code"], session=session)
|
||||
for offset in range(0, len(stale_updates), 500):
|
||||
session.execute(update(StockIndustry), stale_updates[offset:offset + 500])
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _source_session(source: str):
|
||||
"""Serialize all membership state transitions on a pinned MySQL connection."""
|
||||
engine = get_engine()
|
||||
namespace = f"{engine.url.database}:{source}"
|
||||
lock_name = "stock_industry:" + hashlib.sha256(namespace.encode()).hexdigest()[:48]
|
||||
with engine.connect() as connection:
|
||||
acquired = connection.scalar(text("SELECT GET_LOCK(:name, 30)"), {"name": lock_name})
|
||||
connection.commit()
|
||||
if acquired != 1:
|
||||
raise RuntimeError(f"Could not acquire membership sync lock for {source}")
|
||||
try:
|
||||
with Session(bind=connection) as session:
|
||||
yield session
|
||||
finally:
|
||||
connection.execute(text("SELECT RELEASE_LOCK(:name)"), {"name": lock_name})
|
||||
connection.commit()
|
||||
|
||||
|
||||
def deactivate_stock_industry_sources(sources: list[str]) -> None:
|
||||
"""Retire sources without removing their rows or rewriting their industry codes."""
|
||||
for source in sorted(set(sources)):
|
||||
with _source_session(source) as session:
|
||||
while True:
|
||||
ids = list(session.scalars(
|
||||
select(StockIndustry.id)
|
||||
.where(StockIndustry.source == source, StockIndustry.active.is_(True))
|
||||
.order_by(StockIndustry.id).limit(500)
|
||||
))
|
||||
if not ids:
|
||||
break
|
||||
session.execute(update(StockIndustry).where(StockIndustry.id.in_(ids)).values(active=False))
|
||||
session.commit()
|
||||
|
||||
|
||||
def replace_stock_industry_memberships(rows: list[dict]) -> None:
|
||||
"""Replace only the (source, stock) sets present in rows; omitted stocks stay intact.
|
||||
|
||||
Callers must pass all matches together, exclude failed fetches, and supply their
|
||||
unmatched sentinel when classification succeeds without any matching node.
|
||||
All membership writers share a database-scoped MySQL advisory lock per source.
|
||||
"""
|
||||
by_source = defaultdict(lambda: defaultdict(dict))
|
||||
for row in rows:
|
||||
if not all(row.get(key) for key in ("source", "stock_code", "industry_code", "updated_at")):
|
||||
raise ValueError("Membership requires source, stock_code, industry_code and updated_at")
|
||||
by_source[row["source"]][row["stock_code"]][row["industry_code"]] = {
|
||||
key: row.get(key)
|
||||
for key in ("source", "stock_code", "industry_code", "effective_date", "updated_at")
|
||||
}
|
||||
if not by_source:
|
||||
return
|
||||
for source, stocks in sorted(by_source.items()):
|
||||
with _source_session(source) as session:
|
||||
codes = sorted(stocks)
|
||||
for offset in range(0, len(codes), 100):
|
||||
batch = [row for code in codes[offset:offset + 100] for _, row in sorted(stocks[code].items())]
|
||||
_replace_membership_batch(session, batch)
|
||||
session.commit()
|
||||
@@ -0,0 +1,221 @@
|
||||
"""A 股数据的 SQLAlchemy ORM 模型定义。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy import Boolean, Column, Date, DateTime, Float, Index, Integer, String, Text, UniqueConstraint
|
||||
from sqlalchemy.orm import DeclarativeBase
|
||||
|
||||
|
||||
class Base(DeclarativeBase):
|
||||
pass
|
||||
|
||||
|
||||
class StockInfo(Base):
|
||||
__tablename__ = "stock_info"
|
||||
|
||||
code = Column(String(10), primary_key=True, comment="股票代码")
|
||||
name = Column(String(50), comment="股票名称")
|
||||
ipo_date = Column(Date, comment="上市日期")
|
||||
|
||||
|
||||
class StockDaily(Base):
|
||||
__tablename__ = "stock_daily"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_daily_code_date"),
|
||||
Index("ix_daily_date", "date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
open = Column(Float, nullable=False, comment="开盘价")
|
||||
close = Column(Float, nullable=False, comment="收盘价")
|
||||
high = Column(Float, nullable=False, comment="最高价")
|
||||
low = Column(Float, nullable=False, comment="最低价")
|
||||
volume = Column(Float, nullable=False, comment="成交量")
|
||||
turnover = Column(Float, nullable=False, comment="成交额")
|
||||
amplitude = Column(Float, nullable=False, comment="振幅%")
|
||||
pct_change = Column(Float, nullable=False, comment="涨跌幅%")
|
||||
change = Column(Float, nullable=False, comment="涨跌额")
|
||||
turnover_rate = Column(Float, nullable=False, comment="换手率%")
|
||||
|
||||
|
||||
class IndexDaily(Base):
|
||||
__tablename__ = "index_daily"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_index_code_date"),
|
||||
Index("ix_index_date", "date"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="指数代码")
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
pct_change = Column(Float, comment="涨跌幅%")
|
||||
|
||||
|
||||
class IndexMin5(Base):
|
||||
"""指数 5 分钟行情，与股票分钟行情分表避免同代码冲突。"""
|
||||
|
||||
__tablename__ = "index_min5"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "datetime", name="uq_index_min5_code_dt"),
|
||||
Index("ix_index_min5_datetime", "datetime"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="指数代码")
|
||||
datetime = Column(DateTime, nullable=False, comment="时间")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
|
||||
|
||||
class TradingDay(Base):
|
||||
__tablename__ = "trading_day"
|
||||
__table_args__ = (UniqueConstraint("date", name="uq_trading_day_date"),)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
|
||||
|
||||
class StockTradeStatus(Base):
|
||||
__tablename__ = "stock_trade_status"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "date", name="uq_trade_status_code_date"),
|
||||
Index("ix_trade_status_date", "date"),
|
||||
Index("ix_trade_status_status", "status"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
date = Column(Date, nullable=False, comment="交易日期")
|
||||
status = Column(String(20), nullable=False, comment="交易状态")
|
||||
source = Column(String(20), nullable=False, comment="数据源")
|
||||
daily_exists = Column(Integer, nullable=False, comment="日K是否存在")
|
||||
min5_count = Column(Integer, nullable=False, comment="5分钟K数量")
|
||||
reason = Column(String(255), comment="判定原因")
|
||||
checked_at = Column(DateTime, nullable=False, comment="检查时间")
|
||||
|
||||
|
||||
class StockMin5(Base):
|
||||
__tablename__ = "stock_min5"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("code", "datetime", name="uq_min5_code_dt"),
|
||||
Index("ix_min5_datetime", "datetime"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
code = Column(String(10), nullable=False, comment="股票代码")
|
||||
datetime = Column(DateTime, nullable=False, comment="时间")
|
||||
open = Column(Float, comment="开盘价")
|
||||
high = Column(Float, comment="最高价")
|
||||
low = Column(Float, comment="最低价")
|
||||
close = Column(Float, comment="收盘价")
|
||||
volume = Column(Float, comment="成交量")
|
||||
amount = Column(Float, comment="成交额")
|
||||
|
||||
|
||||
class StockSector(Base):
|
||||
__tablename__ = "stock_sector"
|
||||
__table_args__ = (UniqueConstraint("code", name="uq_sector_code"),)
|
||||
|
||||
code = Column(String(10), primary_key=True, comment="股票代码")
|
||||
industry = Column(String(100), comment="行业分类")
|
||||
region = Column(String(20), comment="省份/地域")
|
||||
|
||||
|
||||
class IndustryNode(Base):
|
||||
"""标准行业分类节点。行业树和概念板块分开存储。"""
|
||||
|
||||
__tablename__ = "industry_node"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("source", "industry_code", name="uq_industry_node_source_code"),
|
||||
Index("ix_industry_node_source_parent", "source", "parent_code"),
|
||||
Index("ix_industry_node_source_active_level", "source", "active", "industry_level"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
source = Column(String(20), nullable=False, comment="分类来源")
|
||||
industry_code = Column(String(32), nullable=False, comment="行业代码")
|
||||
industry_name = Column(String(100), nullable=False, comment="行业名称")
|
||||
parent_code = Column(String(32), nullable=False, default="", comment="父行业代码，根节点为空")
|
||||
industry_level = Column(Integer, nullable=False, comment="来源定义的行业层级")
|
||||
active = Column(Boolean, nullable=False, default=True, comment="当前分类是否有效")
|
||||
updated_at = Column(DateTime, nullable=False, comment="同步时间")
|
||||
|
||||
|
||||
class StockIndustry(Base):
|
||||
"""股票当前行业归属；同一来源下可归属多个节点，同一节点不重复。"""
|
||||
|
||||
__tablename__ = "stock_industry"
|
||||
__table_args__ = (
|
||||
UniqueConstraint("source", "stock_code", "industry_code", name="uq_stock_industry_source_stock_industry"),
|
||||
Index("ix_stock_industry_source_industry", "source", "industry_code"),
|
||||
Index("ix_stock_industry_source_active", "source", "active", "id"),
|
||||
)
|
||||
|
||||
id = Column(Integer, primary_key=True, autoincrement=True)
|
||||
source = Column(String(20), nullable=False, comment="分类来源")
|
||||
stock_code = Column(String(10), nullable=False, comment="股票代码")
|
||||
industry_code = Column(String(32), nullable=False, comment="当前行业代码")
|
||||
active = Column(Boolean, nullable=False, default=True, server_default="1", comment="当前归属是否有效；失效记录保留")
|
||||
effective_date = Column(Date, comment="分类生效日期")
|
||||
updated_at = Column(DateTime, nullable=False, comment="同步时间")
|
||||
|
||||
|
||||
class StockF10Evidence(Base):
|
||||
"""同花顺 F10 分类证据缓存，避免产业链刷新时重复抓取。"""
|
||||
|
||||
__tablename__ = "stock_f10_evidence"
|
||||
|
||||
stock_code = Column(String(10), primary_key=True, comment="股票代码")
|
||||
evidence_text = Column(Text, nullable=False, comment="概念名称与概念解析")
|
||||
fetched_at = Column(DateTime, nullable=False, index=True, comment="抓取时间")
|
||||
|
||||
|
||||
STOCK_MINUTE_MODELS = {
|
||||
5: StockMin5,
|
||||
}
|
||||
|
||||
|
||||
def normalize_minute_freq(freq: int | str) -> int:
|
||||
text = str(freq).strip().lower().replace("分钟", "").replace("min", "").rstrip("m")
|
||||
try:
|
||||
value = int(text)
|
||||
except (TypeError, ValueError):
|
||||
raise ValueError(f"不支持的分钟周期: {freq}") from None
|
||||
if value not in STOCK_MINUTE_MODELS:
|
||||
raise ValueError(f"不支持的分钟周期: {freq}，仅支持 5m")
|
||||
return value
|
||||
|
||||
|
||||
def get_stock_minute_model(freq: int | str) -> type[Base]:
|
||||
return STOCK_MINUTE_MODELS[normalize_minute_freq(freq)]
|
||||
|
||||
|
||||
__all__ = [
|
||||
"Base",
|
||||
"StockInfo",
|
||||
"StockDaily",
|
||||
"IndexDaily",
|
||||
"IndexMin5",
|
||||
"TradingDay",
|
||||
"StockTradeStatus",
|
||||
"StockMin5",
|
||||
"StockSector",
|
||||
"IndustryNode",
|
||||
"StockIndustry",
|
||||
"StockF10Evidence",
|
||||
"STOCK_MINUTE_MODELS",
|
||||
"normalize_minute_freq",
|
||||
"get_stock_minute_model",
|
||||
]
|
||||
@@ -0,0 +1,147 @@
|
||||
"""数据库连接、缓存和批量写入能力。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.dialects.mysql import insert as mysql_insert
|
||||
from sqlalchemy.orm import Session, sessionmaker
|
||||
|
||||
from src.config import get_fetch_config, get_mysql_url
|
||||
from src.db.models import Base, StockInfo
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("db")
|
||||
|
||||
_engine = None
|
||||
_SessionFactory = None
|
||||
_stock_codes_cache: list[str] | None = None
|
||||
_ipo_dates_cache: dict[str, str] | None = None
|
||||
|
||||
|
||||
def get_engine():
|
||||
global _engine
|
||||
if _engine is None:
|
||||
_engine = create_engine(get_mysql_url(), pool_size=5, pool_recycle=3600)
|
||||
return _engine
|
||||
|
||||
|
||||
def get_session() -> Session:
|
||||
global _SessionFactory
|
||||
if _SessionFactory is None:
|
||||
_SessionFactory = sessionmaker(bind=get_engine())
|
||||
return _SessionFactory()
|
||||
|
||||
|
||||
def init_db() -> None:
|
||||
Base.metadata.create_all(get_engine())
|
||||
_logger.info("数据库表初始化完成")
|
||||
|
||||
|
||||
def get_stock_codes() -> list[str]:
|
||||
global _stock_codes_cache
|
||||
if _stock_codes_cache is None:
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(select(StockInfo.code).order_by(StockInfo.code))
|
||||
_stock_codes_cache = [row[0] for row in result]
|
||||
finally:
|
||||
session.close()
|
||||
return _stock_codes_cache
|
||||
|
||||
|
||||
def invalidate_stock_codes_cache() -> None:
|
||||
global _stock_codes_cache
|
||||
_stock_codes_cache = None
|
||||
|
||||
|
||||
def invalidate_ipo_dates_cache() -> None:
|
||||
global _ipo_dates_cache
|
||||
_ipo_dates_cache = None
|
||||
|
||||
|
||||
def get_ipo_dates() -> dict[str, str]:
|
||||
global _ipo_dates_cache
|
||||
if _ipo_dates_cache is None:
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(select(StockInfo.code, StockInfo.ipo_date))
|
||||
_ipo_dates_cache = {code: str(ipo) for code, ipo in result if ipo}
|
||||
finally:
|
||||
session.close()
|
||||
return _ipo_dates_cache
|
||||
|
||||
|
||||
def _get_upsert_batch_size() -> int:
|
||||
try:
|
||||
value = int(get_fetch_config().get("db_batch_size", 1000))
|
||||
except Exception:
|
||||
return 1000
|
||||
return max(value, 1)
|
||||
|
||||
|
||||
def _iter_batches(rows: list[dict], batch_size: int):
|
||||
for index in range(0, len(rows), batch_size):
|
||||
batch = rows[index:index + batch_size]
|
||||
if batch:
|
||||
yield batch
|
||||
|
||||
|
||||
def _upsert_batches(session: Session, model_cls: type[Base], rows: list[dict], index_columns: list[str]) -> None:
|
||||
"""Use the caller's session so related writes can commit as one bounded unit."""
|
||||
for batch in _iter_batches(rows, _get_upsert_batch_size()):
|
||||
# SQLite supports isolated persistence tests without connecting to the application DB.
|
||||
if session.get_bind().dialect.name == "sqlite":
|
||||
from sqlalchemy.dialects.sqlite import insert as sqlite_insert
|
||||
statement = sqlite_insert(model_cls).values(batch)
|
||||
inserted = statement.excluded
|
||||
else:
|
||||
statement = mysql_insert(model_cls).values(batch)
|
||||
inserted = statement.inserted
|
||||
input_keys = set(batch[0])
|
||||
update_values = {
|
||||
column.name: inserted[column.name]
|
||||
for column in model_cls.__table__.columns
|
||||
if column.name in input_keys
|
||||
and column.name not in index_columns
|
||||
and not column.primary_key
|
||||
}
|
||||
if session.get_bind().dialect.name == "sqlite":
|
||||
statement = (
|
||||
statement.on_conflict_do_update(index_elements=index_columns, set_=update_values)
|
||||
if update_values else statement.on_conflict_do_nothing(index_elements=index_columns)
|
||||
)
|
||||
else:
|
||||
statement = statement.on_duplicate_key_update(
|
||||
**(update_values or {index_columns[0]: inserted[index_columns[0]]})
|
||||
)
|
||||
session.execute(statement)
|
||||
|
||||
|
||||
def batch_upsert(model_cls: type[Base], rows: list[dict], index_columns: list[str], *, session: Session | None = None) -> None:
|
||||
if not rows:
|
||||
return
|
||||
if session is not None:
|
||||
_upsert_batches(session, model_cls, rows, index_columns)
|
||||
return
|
||||
|
||||
session = get_session()
|
||||
try:
|
||||
_upsert_batches(session, model_cls, rows, index_columns)
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
__all__ = [
|
||||
"get_engine",
|
||||
"get_session",
|
||||
"init_db",
|
||||
"batch_upsert",
|
||||
"get_stock_codes",
|
||||
"get_ipo_dates",
|
||||
"invalidate_stock_codes_cache",
|
||||
"invalidate_ipo_dates_cache",
|
||||
]
|
||||
@@ -0,0 +1,59 @@
|
||||
"""交易状态标记的只读查询。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from src.db.models import StockTradeStatus
|
||||
from src.db.runtime import get_session
|
||||
|
||||
SUSPENDED_STATUS = "suspended"
|
||||
QUERY_BATCH_SIZE = 500
|
||||
|
||||
|
||||
def _parse_day(value: str) -> date:
|
||||
text = str(value).strip()
|
||||
if len(text) == 8 and text.isdigit():
|
||||
return datetime.strptime(text, "%Y%m%d").date()
|
||||
return datetime.strptime(text[:10], "%Y-%m-%d").date()
|
||||
|
||||
|
||||
def _chunks(values: list[str], batch_size: int = QUERY_BATCH_SIZE):
|
||||
for index in range(0, len(values), batch_size):
|
||||
batch = values[index:index + batch_size]
|
||||
if batch:
|
||||
yield batch
|
||||
|
||||
|
||||
def get_suspended_trade_days(
|
||||
codes: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
) -> dict[str, set[str]]:
|
||||
"""返回区间内已归档为停牌的交易日，按股票代码分组。"""
|
||||
if not codes:
|
||||
return {}
|
||||
|
||||
start_day = _parse_day(start_date)
|
||||
end_day = _parse_day(end_date)
|
||||
result_by_code: dict[str, set[str]] = {}
|
||||
session = get_session()
|
||||
try:
|
||||
for batch in _chunks(list(dict.fromkeys(codes))):
|
||||
rows = session.execute(
|
||||
select(StockTradeStatus.code, StockTradeStatus.date)
|
||||
.where(StockTradeStatus.code.in_(batch))
|
||||
.where(StockTradeStatus.date >= start_day)
|
||||
.where(StockTradeStatus.date <= end_day)
|
||||
.where(StockTradeStatus.status == SUSPENDED_STATUS)
|
||||
)
|
||||
for code, day in rows:
|
||||
result_by_code.setdefault(str(code), set()).add(str(day))
|
||||
finally:
|
||||
session.close()
|
||||
return result_by_code
|
||||
|
||||
|
||||
__all__ = ["SUSPENDED_STATUS", "get_suspended_trade_days"]
|
||||
@@ -0,0 +1,316 @@
|
||||
"""CNINFO 巨潮产业细分树与股票当前产业归属同步。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
from sqlalchemy import select, update
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import IndustryNode, StockIndustry, batch_upsert, get_session, get_stock_codes
|
||||
from src.db.industry import deactivate_stock_industry_sources, replace_stock_industry_memberships
|
||||
from src.log import get_logger
|
||||
|
||||
CNINFO_DETAIL_SOURCE = "cninfo_xc_detail"
|
||||
CNINFO_DETAIL_TAXONOMY = "巨潮产业细分标准"
|
||||
CNINFO_DETAIL_TYPE_CODE = "008006"
|
||||
_LEGACY_SW_SOURCE = "cninfo_sw"
|
||||
|
||||
_logger = get_logger("cninfo.industry")
|
||||
|
||||
|
||||
def _config_number(key: str, default: float) -> float:
|
||||
try:
|
||||
return float(get_fetch_config().get(key, default))
|
||||
except Exception:
|
||||
return default
|
||||
|
||||
|
||||
def _text(value: Any) -> str:
|
||||
if value is None:
|
||||
return ""
|
||||
value = str(value).strip()
|
||||
return "" if value.lower() in {"", "nan", "nat", "none"} else value
|
||||
|
||||
|
||||
def _first_value(record: dict[str, Any], names: tuple[str, ...]) -> str:
|
||||
for name in names:
|
||||
value = _text(record.get(name))
|
||||
if value:
|
||||
return value
|
||||
return ""
|
||||
|
||||
|
||||
def parse_cninfo_taxonomy(
|
||||
frame,
|
||||
*,
|
||||
source: str,
|
||||
now: datetime | None = None,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""把 AkShare 的 CNINFO 行业目录 DataFrame 规范化为行业节点行。"""
|
||||
checked_at = now or datetime.now()
|
||||
raw_rows = frame.to_dict("records")
|
||||
parsed: list[dict[str, Any]] = []
|
||||
for record in raw_rows:
|
||||
industry_code = _first_value(record, ("类目编码", "类目编号", "类别编号", "项目编号", "行业编码", "行业代码"))
|
||||
industry_name = _first_value(record, ("类目名称", "项目名称", "行业名称"))
|
||||
parent_code = _first_value(record, ("父类编码", "父级编码", "父行业代码"))
|
||||
level_text = _first_value(record, ("分级", "行业层级", "层级"))
|
||||
if not industry_code or not industry_name:
|
||||
continue
|
||||
try:
|
||||
industry_level = int(float(level_text))
|
||||
except (TypeError, ValueError):
|
||||
industry_level = max((len(industry_code) - 1) // 2, 0)
|
||||
parsed.append(
|
||||
{
|
||||
"source": source,
|
||||
"industry_code": industry_code,
|
||||
"industry_name": industry_name,
|
||||
"parent_code": parent_code,
|
||||
"industry_level": industry_level,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
)
|
||||
|
||||
codes = {row["industry_code"] for row in parsed}
|
||||
for row in parsed:
|
||||
if row["parent_code"] not in codes:
|
||||
row["parent_code"] = ""
|
||||
return sorted(parsed, key=lambda row: (row["industry_level"], row["industry_code"]))
|
||||
|
||||
|
||||
def _normalized_name(value: str) -> str:
|
||||
return "".join(value.lower().split())
|
||||
|
||||
|
||||
def parse_cninfo_profile_industry(
|
||||
profile_frame,
|
||||
stock_code: str,
|
||||
nodes: list[dict[str, Any]],
|
||||
*,
|
||||
now: datetime | None = None,
|
||||
) -> dict[str, Any] | None:
|
||||
"""按 CNINFO 公司概况把股票保守归到巨潮产业细分节点。"""
|
||||
records = profile_frame.to_dict("records")
|
||||
if not records:
|
||||
return None
|
||||
record = records[0]
|
||||
source_industry = _first_value(record, ("所属行业",))
|
||||
main_business = _first_value(record, ("主营业务",))
|
||||
business_scope = _first_value(record, ("经营范围",))
|
||||
if not source_industry and not main_business:
|
||||
return None
|
||||
|
||||
nodes_by_code = {row["industry_code"]: row for row in nodes}
|
||||
nodes_by_name: dict[str, list[dict[str, Any]]] = {}
|
||||
for node in nodes:
|
||||
nodes_by_name.setdefault(_normalized_name(node["industry_name"]), []).append(node)
|
||||
base_candidates = nodes_by_name.get(_normalized_name(source_industry), [])
|
||||
if not base_candidates:
|
||||
return None
|
||||
deepest_level = max(row["industry_level"] for row in base_candidates)
|
||||
deepest_candidates = [row for row in base_candidates if row["industry_level"] == deepest_level]
|
||||
if len(deepest_candidates) != 1:
|
||||
return None
|
||||
base = deepest_candidates[0]
|
||||
|
||||
def is_descendant(code: str, ancestor_code: str) -> bool:
|
||||
seen: set[str] = set()
|
||||
while code and code not in seen:
|
||||
if code == ancestor_code:
|
||||
return True
|
||||
seen.add(code)
|
||||
code = nodes_by_code.get(code, {}).get("parent_code", "")
|
||||
return False
|
||||
|
||||
main_text = main_business.lower()
|
||||
scope_text = business_scope.lower()
|
||||
scores: dict[str, int] = {}
|
||||
for code, keywords in CNINFO_DETAIL_PROFILE_RULES.items():
|
||||
if code not in nodes_by_code or not is_descendant(code, base["industry_code"]):
|
||||
continue
|
||||
score = max((main_text.count(keyword) * 3 + scope_text.count(keyword) for keyword in keywords), default=0)
|
||||
if score > 0:
|
||||
scores[code] = max(scores.get(code, 0), score)
|
||||
for node in nodes:
|
||||
name = node["industry_name"].lower()
|
||||
if len(name) < 4 or not is_descendant(node["industry_code"], base["industry_code"]):
|
||||
continue
|
||||
score = main_text.count(name) * 3 + scope_text.count(name)
|
||||
if score > 0:
|
||||
code = node["industry_code"]
|
||||
scores[code] = max(scores.get(code, 0), score)
|
||||
|
||||
industry_code = base["industry_code"]
|
||||
if scores:
|
||||
scored = sorted(((score, code) for code, score in scores.items()), reverse=True)
|
||||
best_score, best_code = scored[0]
|
||||
second_score = scored[1][0] if len(scored) > 1 else 0
|
||||
if best_score >= 2 and best_score > second_score:
|
||||
industry_code = best_code
|
||||
return {
|
||||
"source": CNINFO_DETAIL_SOURCE,
|
||||
"stock_code": stock_code,
|
||||
"industry_code": industry_code,
|
||||
"effective_date": None,
|
||||
"updated_at": now or datetime.now(),
|
||||
}
|
||||
|
||||
|
||||
# 规则仅把 CNINFO 公司披露中的产品词映射到对应的 CNINFO 正式细分节点。
|
||||
CNINFO_DETAIL_PROFILE_RULES = {
|
||||
"XC37050201": ("航天飞机",),
|
||||
"XC37050202": ("航空飞机", "飞机整机", "飞机制造", "无人机整机"),
|
||||
"XC37050203": ("卫星发射", "运载火箭", "火箭发射", "卫星发射装置"),
|
||||
"XC37050204": ("航空器零部件", "航空航天器零配件", "飞机零部件", "航空发动机", "航空电子", "航电", "机体结构"),
|
||||
}
|
||||
|
||||
|
||||
def _with_retries(operation, *, attempts: int, delay_seconds: float, description: str):
|
||||
last_error: Exception | None = None
|
||||
for attempt in range(1, max(attempts, 1) + 1):
|
||||
try:
|
||||
return operation()
|
||||
except Exception as exc: # CNINFO occasionally returns an empty throttling response.
|
||||
last_error = exc
|
||||
if attempt < max(attempts, 1):
|
||||
time.sleep(max(delay_seconds, 0.1) * attempt)
|
||||
assert last_error is not None
|
||||
raise RuntimeError(f"{description} 失败: {last_error}") from last_error
|
||||
|
||||
|
||||
def fetch_cninfo_detail_taxonomy(*, attempts: int, delay_seconds: float):
|
||||
import akshare as ak
|
||||
|
||||
return _with_retries(
|
||||
lambda: ak.stock_industry_category_cninfo(symbol=CNINFO_DETAIL_TAXONOMY),
|
||||
attempts=attempts,
|
||||
delay_seconds=delay_seconds,
|
||||
description="CNINFO 巨潮产业细分目录",
|
||||
)
|
||||
|
||||
|
||||
def fetch_cninfo_stock_profile(stock_code: str, *, attempts: int, delay_seconds: float):
|
||||
import akshare as ak
|
||||
|
||||
return _with_retries(
|
||||
lambda: ak.stock_profile_cninfo(symbol=stock_code),
|
||||
attempts=attempts,
|
||||
delay_seconds=delay_seconds,
|
||||
description=f"CNINFO {stock_code} 公司概况",
|
||||
)
|
||||
|
||||
|
||||
def _mark_nodes_inactive(source: str) -> None:
|
||||
session = get_session()
|
||||
try:
|
||||
ids = list(session.scalars(select(IndustryNode.id).where(IndustryNode.source == source, IndustryNode.active.is_(True))))
|
||||
for offset in range(0, len(ids), 500):
|
||||
session.execute(update(IndustryNode).where(IndustryNode.id.in_(ids[offset:offset + 500])).values(active=False))
|
||||
session.commit()
|
||||
except Exception:
|
||||
session.rollback()
|
||||
raise
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _existing_stock_codes(source: str) -> set[str]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(StockIndustry.stock_code).where(StockIndustry.source == source, StockIndustry.active.is_(True))
|
||||
)
|
||||
return {str(row[0]) for row in rows}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _deactivate_source(source: str) -> None:
|
||||
_mark_nodes_inactive(source)
|
||||
deactivate_stock_industry_sources([source])
|
||||
|
||||
|
||||
def _parse_symbol_list(symbol: str | None) -> list[str]:
|
||||
if not symbol:
|
||||
return []
|
||||
return sorted({part.strip().upper().replace("SH", "").replace("SZ", "").replace("BJ", "") for part in symbol.split(",") if part.strip()})
|
||||
|
||||
|
||||
def sync_cninfo_detail_industry_tree(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
taxonomy_only: bool = False,
|
||||
refresh: bool = False,
|
||||
delay_seconds: float | None = None,
|
||||
) -> tuple[int, int, int]:
|
||||
"""同步产业目录和当前股票归属；可安全重复运行以继续补齐失败股票。"""
|
||||
delay = max(delay_seconds if delay_seconds is not None else _config_number("industry_delay", 0.25), 0)
|
||||
attempts = max(int(_config_number("industry_retry", 3)), 1)
|
||||
checked_at = datetime.now()
|
||||
|
||||
taxonomy_frame = fetch_cninfo_detail_taxonomy(attempts=attempts, delay_seconds=delay)
|
||||
nodes = parse_cninfo_taxonomy(taxonomy_frame, source=CNINFO_DETAIL_SOURCE, now=checked_at)
|
||||
if not nodes:
|
||||
raise RuntimeError("CNINFO 未返回有效的巨潮产业细分目录")
|
||||
_mark_nodes_inactive(CNINFO_DETAIL_SOURCE)
|
||||
batch_upsert(IndustryNode, nodes, ["source", "industry_code"])
|
||||
_deactivate_source(_LEGACY_SW_SOURCE)
|
||||
_logger.info("CNINFO 巨潮产业细分目录同步完成，节点:%d", len(nodes))
|
||||
|
||||
if taxonomy_only:
|
||||
return len(nodes), 0, 0
|
||||
|
||||
requested_codes = _parse_symbol_list(symbol)
|
||||
all_codes = requested_codes or get_stock_codes()
|
||||
existing_codes = set() if refresh else _existing_stock_codes(CNINFO_DETAIL_SOURCE)
|
||||
target_codes = [code for code in all_codes if code not in existing_codes]
|
||||
if limit is not None:
|
||||
target_codes = target_codes[: max(int(limit), 0)]
|
||||
if not target_codes:
|
||||
_logger.info("CNINFO 股票行业归属无需同步")
|
||||
return len(nodes), 0, 0
|
||||
|
||||
mapped = 0
|
||||
failed = 0
|
||||
pending_rows: list[dict[str, Any]] = []
|
||||
flush_size = 100
|
||||
for index, stock_code in enumerate(target_codes, start=1):
|
||||
try:
|
||||
profile = fetch_cninfo_stock_profile(stock_code, attempts=attempts, delay_seconds=delay)
|
||||
row = parse_cninfo_profile_industry(profile, stock_code, nodes, now=checked_at)
|
||||
if row is None:
|
||||
failed += 1
|
||||
_logger.warning("[%d/%d] %s 无法由 CNINFO 披露信息确认产业归属", index, len(target_codes), stock_code)
|
||||
else:
|
||||
pending_rows.append(row)
|
||||
mapped += 1
|
||||
except Exception as exc:
|
||||
failed += 1
|
||||
_logger.warning("[%d/%d] %s 产业归属同步失败: %s", index, len(target_codes), stock_code, exc)
|
||||
if len(pending_rows) >= flush_size:
|
||||
replace_stock_industry_memberships(pending_rows)
|
||||
pending_rows = []
|
||||
if delay > 0 and index < len(target_codes):
|
||||
time.sleep(delay)
|
||||
if index % 25 == 0 or index == len(target_codes):
|
||||
_logger.info("CNINFO 股票产业归属进度 %d/%d，成功:%d，失败:%d", index, len(target_codes), mapped, failed)
|
||||
|
||||
if pending_rows:
|
||||
replace_stock_industry_memberships(pending_rows)
|
||||
return len(nodes), mapped, failed
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CNINFO_DETAIL_SOURCE",
|
||||
"CNINFO_DETAIL_TAXONOMY",
|
||||
"CNINFO_DETAIL_TYPE_CODE",
|
||||
"parse_cninfo_taxonomy",
|
||||
"parse_cninfo_profile_industry",
|
||||
"sync_cninfo_detail_industry_tree",
|
||||
]
|
||||
+12
-757
@@ -1,769 +1,24 @@
|
||||
"""日线行情抓取模块 - BaoStock。"""
|
||||
"""兼容入口：日线行情统一使用当前行情实现。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import baostock as bs
|
||||
import pandas as pd
|
||||
|
||||
from src.baostock_conn import bs_query, code_to_bs
|
||||
from src.config import get_fetch_config
|
||||
from src.db import (
|
||||
StockDaily,
|
||||
StockInfo,
|
||||
StockNoData,
|
||||
TradingDay,
|
||||
batch_upsert,
|
||||
get_ipo_dates,
|
||||
get_session,
|
||||
get_stock_codes,
|
||||
invalidate_ipo_dates_cache,
|
||||
invalidate_stock_codes_cache,
|
||||
)
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import select, text
|
||||
|
||||
_logger = get_logger("daily")
|
||||
ak = None
|
||||
|
||||
|
||||
def _clean(val):
|
||||
if val is None:
|
||||
return None
|
||||
if isinstance(val, str) and val.strip() == "":
|
||||
return None
|
||||
return val
|
||||
|
||||
|
||||
def _to_float(val):
|
||||
val = _clean(val)
|
||||
if val is None:
|
||||
return None
|
||||
try:
|
||||
return float(val)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
|
||||
|
||||
def _parse_ipo_date(raw: str | None):
|
||||
if not raw:
|
||||
return None
|
||||
raw = str(raw).strip()
|
||||
if not raw:
|
||||
return None
|
||||
if len(raw) >= 10 and raw[4] == "-" and raw[7] == "-":
|
||||
try:
|
||||
return datetime.strptime(raw[:10], "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
return None
|
||||
if len(raw) == 8 and raw.isdigit():
|
||||
try:
|
||||
return datetime.strptime(raw, "%Y%m%d").date()
|
||||
except ValueError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _fill_derived_fields(rows: list[dict]) -> list[dict]:
|
||||
"""补算缺失的振幅、涨跌幅、涨跌额。"""
|
||||
if not rows:
|
||||
return rows
|
||||
for i, row in enumerate(rows):
|
||||
close = row.get("close")
|
||||
high = row.get("high")
|
||||
low = row.get("low")
|
||||
preclose = rows[i - 1]["close"] if i > 0 else None
|
||||
|
||||
if close is not None and preclose is not None and float(preclose) != 0:
|
||||
pc = float(preclose)
|
||||
c = float(close)
|
||||
if row.get("pct_change") is None:
|
||||
row["pct_change"] = round((c - pc) / pc * 100, 2)
|
||||
if row.get("change") is None:
|
||||
row["change"] = round(c - pc, 3)
|
||||
if row.get("amplitude") is None and high is not None and low is not None:
|
||||
row["amplitude"] = round((float(high) - float(low)) / pc * 100, 2)
|
||||
return rows
|
||||
|
||||
|
||||
def _is_complete_daily_row(row: dict) -> bool:
|
||||
"""判断日线行是否满足 stock_daily 的全字段非空要求。"""
|
||||
required_fields = (
|
||||
"open",
|
||||
"close",
|
||||
"high",
|
||||
"low",
|
||||
"volume",
|
||||
"turnover",
|
||||
"amplitude",
|
||||
"pct_change",
|
||||
"change",
|
||||
"turnover_rate",
|
||||
)
|
||||
return all(row.get(field) is not None for field in required_fields)
|
||||
|
||||
|
||||
def _normalize_code(code: str) -> str:
|
||||
"""把输入代码统一成数据库里的纯 6 位代码。"""
|
||||
code = (code or "").strip()
|
||||
if not code:
|
||||
return code
|
||||
upper = code.upper()
|
||||
if "." in upper:
|
||||
base, suffix = upper.split(".", 1)
|
||||
if suffix in {"SH", "SZ", "BJ"}:
|
||||
return base
|
||||
lower = code.lower()
|
||||
if lower.startswith(("sh", "sz", "bj")) and len(lower) > 2:
|
||||
return lower[2:]
|
||||
return code
|
||||
|
||||
|
||||
def _is_supported_daily_code(code: str) -> bool:
|
||||
"""BaoStock 日线当前只稳定支持沪深 A 股代码。"""
|
||||
clean_code = _normalize_code(code)
|
||||
return len(clean_code) == 6 and clean_code.isdigit() and clean_code[0] in {"0", "3", "6"}
|
||||
|
||||
|
||||
def _split_supported_daily_codes(codes: list[str]) -> tuple[list[str], list[str]]:
|
||||
"""把可抓取和应跳过的日线代码分开。"""
|
||||
supported: list[str] = []
|
||||
skipped: list[str] = []
|
||||
for code in codes:
|
||||
if _is_supported_daily_code(code):
|
||||
supported.append(code)
|
||||
else:
|
||||
skipped.append(code)
|
||||
return supported, skipped
|
||||
|
||||
|
||||
def _preview_codes(codes: list[str], limit: int = 5) -> str:
|
||||
"""把代码列表压缩成日志预览字符串。"""
|
||||
if not codes:
|
||||
return "[]"
|
||||
preview = codes[:limit]
|
||||
suffix = ""
|
||||
if len(codes) > limit:
|
||||
suffix = f", ...(+{len(codes) - limit})"
|
||||
return "[" + ", ".join(preview) + suffix + "]"
|
||||
|
||||
|
||||
def _mark_no_data_days(code: str, days: list[str]) -> int:
|
||||
"""把某只股票缺失的交易日写入停牌/无数据表。"""
|
||||
if not days:
|
||||
return 0
|
||||
|
||||
rows = [{"code": code, "date": day} for day in days]
|
||||
batch_upsert(StockNoData, rows, ["code", "date"])
|
||||
return len(rows)
|
||||
|
||||
|
||||
def _get_no_data_dates(start_date: str, end_date: str) -> dict[str, set[str]]:
|
||||
"""读取区间内已标记为停牌/无数据的日期。"""
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(
|
||||
select(StockNoData.code, StockNoData.date)
|
||||
.where(StockNoData.date >= sd)
|
||||
.where(StockNoData.date <= ed)
|
||||
)
|
||||
no_data: dict[str, set[str]] = {}
|
||||
for code, day in result:
|
||||
no_data.setdefault(code, set()).add(str(day))
|
||||
return no_data
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _hist_df_to_rows(code: str, hist_df: pd.DataFrame) -> list[dict]:
|
||||
"""把表格行情转换为数据库行。"""
|
||||
if hist_df is None or hist_df.empty:
|
||||
return []
|
||||
|
||||
rows: list[dict] = []
|
||||
for _, item in hist_df.iterrows():
|
||||
date_value = item.get("日期")
|
||||
if pd.isna(date_value):
|
||||
continue
|
||||
date = pd.to_datetime(date_value, errors="coerce")
|
||||
if pd.isna(date):
|
||||
continue
|
||||
|
||||
row = {"code": code, "date": date.strftime("%Y-%m-%d")}
|
||||
valid_value = False
|
||||
field_map = {
|
||||
"开盘": "open",
|
||||
"收盘": "close",
|
||||
"最高": "high",
|
||||
"最低": "low",
|
||||
"成交量": "volume",
|
||||
"成交额": "turnover",
|
||||
"振幅": "amplitude",
|
||||
"涨跌幅": "pct_change",
|
||||
"涨跌额": "change",
|
||||
"换手率": "turnover_rate",
|
||||
}
|
||||
for cn_field, db_field in field_map.items():
|
||||
value = item.get(cn_field)
|
||||
if pd.isna(value):
|
||||
value = None
|
||||
elif value is not None:
|
||||
value = pd.to_numeric(value, errors="coerce")
|
||||
if pd.isna(value):
|
||||
value = None
|
||||
if value is not None:
|
||||
valid_value = True
|
||||
row[db_field] = value
|
||||
if valid_value and row.get("volume") is not None:
|
||||
rows.append(row)
|
||||
|
||||
rows = _fill_derived_fields(rows)
|
||||
return [row for row in rows if _is_complete_daily_row(row)]
|
||||
|
||||
|
||||
def _ak_hist_to_rows(code: str, hist_df: pd.DataFrame) -> list[dict]:
|
||||
"""兼容旧测试/调用的历史数据表转换函数。"""
|
||||
return _hist_df_to_rows(code, hist_df)
|
||||
|
||||
|
||||
def _bs_result_to_rows(code: str, rs) -> list[dict]:
|
||||
"""把 BaoStock 查询结果整理为数据库行。"""
|
||||
rows: list[dict] = []
|
||||
while rs.error_code == "0" and rs.next():
|
||||
r = rs.get_row_data()
|
||||
preclose = _to_float(r[7])
|
||||
pct_chg = _to_float(r[8])
|
||||
turn = _to_float(r[9])
|
||||
close = _to_float(r[4])
|
||||
high = _to_float(r[2])
|
||||
low = _to_float(r[3])
|
||||
amp = None
|
||||
if high is not None and low is not None and preclose is not None and preclose != 0:
|
||||
amp = round((high - low) / preclose * 100, 2)
|
||||
chg = None
|
||||
if close is not None and preclose is not None:
|
||||
chg = round(close - preclose, 3)
|
||||
|
||||
rows.append({
|
||||
"code": code,
|
||||
"date": r[0],
|
||||
"open": _to_float(r[1]),
|
||||
"high": high,
|
||||
"low": low,
|
||||
"close": close,
|
||||
"volume": _to_float(r[5]),
|
||||
"turnover": _to_float(r[6]),
|
||||
"amplitude": amp,
|
||||
"pct_change": pct_chg,
|
||||
"change": chg,
|
||||
"turnover_rate": turn,
|
||||
})
|
||||
if rows[-1]["volume"] is None:
|
||||
rows.pop()
|
||||
rows = _fill_derived_fields(rows)
|
||||
return [row for row in rows if _is_complete_daily_row(row)]
|
||||
|
||||
|
||||
def _fetch_stock_list_rows(*, retry: int = 3, delay: float = 0.1) -> list[dict]:
|
||||
"""从 BaoStock 获取沪深 A 股代码与名称。"""
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
with bs_query(bs.query_stock_basic, code="", code_name="") as rs:
|
||||
if rs.error_code != "0":
|
||||
raise RuntimeError(rs.error_msg)
|
||||
rows: list[dict] = []
|
||||
while rs.next():
|
||||
code, name, ipo_date, _out_date, sec_type, status = rs.get_row_data()
|
||||
if str(sec_type) != "1" or str(status) != "1":
|
||||
continue
|
||||
base_code = _normalize_code(code)
|
||||
if not _is_supported_daily_code(base_code):
|
||||
continue
|
||||
item = {"code": base_code, "name": str(name).strip()}
|
||||
parsed_ipo = _parse_ipo_date(ipo_date)
|
||||
if parsed_ipo is not None:
|
||||
item["ipo_date"] = parsed_ipo
|
||||
rows.append(item)
|
||||
return rows
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
break
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
wait_seconds += random.uniform(0, min(wait_seconds * 0.2, 0.5))
|
||||
_logger.warning("BaoStock 获取股票列表第 %d 次失败，%.1fs 后重试: %s", attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
if last_exc is not None:
|
||||
_logger.warning("BaoStock 获取股票列表失败: %s", last_exc)
|
||||
return []
|
||||
|
||||
|
||||
def _bootstrap_stock_list(*, retry: int = 3, delay: float = 0.1) -> list[str]:
|
||||
"""从 BaoStock 抓取股票基本资料并保存。"""
|
||||
rows = _fetch_stock_list_rows(retry=retry, delay=delay)
|
||||
if not rows:
|
||||
return []
|
||||
|
||||
batch_upsert(StockInfo, rows, ["code"])
|
||||
invalidate_stock_codes_cache()
|
||||
invalidate_ipo_dates_cache()
|
||||
_logger.info("BaoStock 股票列表已保存，共 %d 只", len(rows))
|
||||
return [row["code"] for row in rows]
|
||||
|
||||
|
||||
def _bootstrap_trading_days(start_date: str, end_date: str, *, retry: int = 3, delay: float = 0.1) -> list[str]:
|
||||
"""从 BaoStock 抓取指定区间的交易日并保存。"""
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
sd = _format_date(start_date)
|
||||
ed = _format_date(end_date)
|
||||
last_exc: Exception | None = None
|
||||
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
with bs_query(bs.query_trade_dates, start_date=sd, end_date=ed) as rs:
|
||||
if rs.error_code != "0":
|
||||
raise RuntimeError(rs.error_msg)
|
||||
days: list[str] = []
|
||||
while rs.next():
|
||||
day, is_trading_day = rs.get_row_data()
|
||||
if str(is_trading_day) == "1":
|
||||
days.append(str(day))
|
||||
if days:
|
||||
batch_upsert(TradingDay, [{"date": d} for d in days], ["date"])
|
||||
_logger.info("BaoStock 交易日历已保存，%d 个交易日", len(days))
|
||||
return days
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
break
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
wait_seconds += random.uniform(0, min(wait_seconds * 0.2, 0.5))
|
||||
_logger.warning("BaoStock 获取交易日历第 %d 次失败，%.1fs 后重试: %s", attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
if last_exc is not None:
|
||||
_logger.warning("BaoStock 获取交易日历失败: %s", last_exc)
|
||||
return []
|
||||
|
||||
|
||||
def _format_date(d: str) -> str:
|
||||
return f"{d[:4]}-{d[4:6]}-{d[6:8]}"
|
||||
|
||||
|
||||
def get_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
"""获取指定范围内的交易日列表。"""
|
||||
sd = _format_date(start_date)
|
||||
ed = _format_date(end_date)
|
||||
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
)
|
||||
days = [str(row[0]) for row in result]
|
||||
|
||||
if days and days[0] <= sd and days[-1] >= ed:
|
||||
return days
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
fetched = _bootstrap_trading_days(start_date, end_date)
|
||||
if not fetched:
|
||||
return days if days else []
|
||||
return sorted(set(days + fetched))
|
||||
|
||||
|
||||
def _ensure_stock_codes(*, retry: int, delay: float) -> list[str]:
|
||||
"""确保股票列表存在，必要时从 BaoStock 初始化。"""
|
||||
codes = get_stock_codes()
|
||||
if codes:
|
||||
return codes
|
||||
codes = _bootstrap_stock_list(retry=retry, delay=delay)
|
||||
return codes
|
||||
|
||||
|
||||
def _fetch_one_stock(code: str, gap_start: str, gap_end: str) -> list[dict]:
|
||||
"""读取单只股票一个缺口区间的日线数据。"""
|
||||
bs_code = code_to_bs(code)
|
||||
if not bs_code:
|
||||
return []
|
||||
|
||||
sd = gap_start.replace("-", "")
|
||||
ed = gap_end.replace("-", "")
|
||||
start_date = _format_date(sd)
|
||||
end_date = _format_date(ed)
|
||||
with bs_query(
|
||||
bs.query_history_k_data_plus,
|
||||
bs_code,
|
||||
"date,open,high,low,close,volume,amount,preclose,pctChg,turn",
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
frequency="d",
|
||||
adjustflag="2",
|
||||
) as rs:
|
||||
if rs.error_code != "0":
|
||||
raise RuntimeError(rs.error_msg)
|
||||
return _bs_result_to_rows(code, rs)
|
||||
|
||||
|
||||
def _fetch_one_stock_with_retry(
|
||||
code: str,
|
||||
gap_start: str,
|
||||
gap_end: str,
|
||||
*,
|
||||
retry: int = 3,
|
||||
delay: float = 0.1,
|
||||
) -> list[dict]:
|
||||
"""带重试的单股日线抓取。"""
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
return _fetch_one_stock(code, gap_start, gap_end)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
raise
|
||||
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
wait_seconds += random.uniform(0, min(wait_seconds * 0.2, 0.5))
|
||||
_logger.warning(
|
||||
"[BaoStock] %s 第 %d 次失败，%.1fs 后重试: %s",
|
||||
code,
|
||||
attempt,
|
||||
wait_seconds,
|
||||
exc,
|
||||
)
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
if last_exc is not None:
|
||||
raise last_exc
|
||||
return []
|
||||
|
||||
|
||||
def _fetch_missing_daily_rows(
|
||||
code: str,
|
||||
missing_days: list[str],
|
||||
*,
|
||||
retry: int,
|
||||
delay: float,
|
||||
) -> list[dict]:
|
||||
"""把范围查询漏掉的交易日按天补抓回来。"""
|
||||
if not missing_days:
|
||||
return []
|
||||
|
||||
patched_rows: list[dict] = []
|
||||
for day in missing_days:
|
||||
day_key = day.replace("-", "")
|
||||
rows = _fetch_one_stock_with_retry(
|
||||
code,
|
||||
day_key,
|
||||
day_key,
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
)
|
||||
if rows:
|
||||
patched_rows.extend(rows)
|
||||
return patched_rows
|
||||
|
||||
|
||||
def _analyze_gaps(codes: list[str], start_date: str, end_date: str,
|
||||
trading_days: list[str]) -> dict[str, list[str]]:
|
||||
"""一条 SQL 统计每只股票每月行情数，与交易日对比找缺口。"""
|
||||
if not trading_days:
|
||||
return {}
|
||||
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
|
||||
t0 = time.time()
|
||||
ipo_dates = get_ipo_dates()
|
||||
_logger.info("[1/3] 上市日期查询完成 %d 只 %.1fs", len(ipo_dates), time.time() - t0)
|
||||
|
||||
t1 = time.time()
|
||||
session = get_session()
|
||||
try:
|
||||
sql = text("""
|
||||
SELECT
|
||||
code,
|
||||
DATE_FORMAT(date, '%Y-%m') AS month,
|
||||
COUNT(*) AS cnt,
|
||||
SUM(
|
||||
CASE
|
||||
WHEN open IS NULL
|
||||
OR close IS NULL
|
||||
OR high IS NULL
|
||||
OR low IS NULL
|
||||
OR volume IS NULL
|
||||
OR turnover IS NULL
|
||||
THEN 1 ELSE 0
|
||||
END
|
||||
) AS incomplete_cnt
|
||||
FROM stock_daily
|
||||
WHERE date >= :sd AND date <= :ed
|
||||
GROUP BY code, DATE_FORMAT(date, '%Y-%m')
|
||||
""")
|
||||
result = session.execute(sql, {"sd": sd, "ed": ed})
|
||||
code_month_cnt: dict[str, dict[str, int]] = {}
|
||||
quality_gap_codes: set[str] = set()
|
||||
for code, month, cnt, incomplete_cnt in result:
|
||||
code_month_cnt.setdefault(code, {})[month] = int(cnt or 0)
|
||||
if int(incomplete_cnt or 0) > 0:
|
||||
quality_gap_codes.add(code)
|
||||
finally:
|
||||
session.close()
|
||||
_logger.info("[2/3] 行情按月统计完成 %.1fs", time.time() - t1)
|
||||
|
||||
no_data_dates = _get_no_data_dates(start_date, end_date)
|
||||
|
||||
t2 = time.time()
|
||||
td_by_month: dict[str, list[str]] = {}
|
||||
for d in trading_days:
|
||||
td_by_month.setdefault(d[:7], []).append(d)
|
||||
|
||||
gap_codes: set[str] = set()
|
||||
for month_key, month_days in sorted(td_by_month.items()):
|
||||
for code in codes:
|
||||
if code in gap_codes:
|
||||
continue
|
||||
if code in quality_gap_codes:
|
||||
gap_codes.add(code)
|
||||
continue
|
||||
ipo = ipo_dates.get(code)
|
||||
expected = [
|
||||
d for d in month_days
|
||||
if (not ipo or d >= ipo) and d not in no_data_dates.get(code, set())
|
||||
]
|
||||
if not expected:
|
||||
continue
|
||||
cnt = code_month_cnt.get(code, {}).get(month_key, 0)
|
||||
if cnt < len(expected):
|
||||
gap_codes.add(code)
|
||||
|
||||
_logger.info(
|
||||
"[3/3] 缺口对比完成 缺口股票:%d 只 %.1fs",
|
||||
len(gap_codes), time.time() - t2,
|
||||
)
|
||||
|
||||
if not gap_codes:
|
||||
return {}
|
||||
|
||||
gaps: dict[str, list[str]] = {}
|
||||
session = get_session()
|
||||
try:
|
||||
for code in gap_codes:
|
||||
ipo = ipo_dates.get(code)
|
||||
expected = [
|
||||
d for d in trading_days
|
||||
if (not ipo or d >= ipo) and d not in no_data_dates.get(code, set())
|
||||
]
|
||||
if not expected:
|
||||
continue
|
||||
|
||||
if code in quality_gap_codes:
|
||||
gaps[code] = [expected[0], expected[-1]]
|
||||
continue
|
||||
|
||||
existing_dates = {
|
||||
str(row[0]) for row in session.execute(
|
||||
select(StockDaily.date)
|
||||
.where(StockDaily.code == code)
|
||||
.where(StockDaily.date >= expected[0])
|
||||
.where(StockDaily.date <= expected[-1])
|
||||
)
|
||||
}
|
||||
missing = [d for d in expected if d not in existing_dates]
|
||||
if missing:
|
||||
gaps[code] = [missing[0], missing[-1]]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
return gaps
|
||||
from src.fetchers.tdx_daily import fetch_tdx_daily
|
||||
|
||||
|
||||
def fetch_daily(
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
*,
|
||||
no_data_only: bool = False,
|
||||
symbol: str | None = None,
|
||||
status_only: bool = False,
|
||||
source: str = "tdx",
|
||||
):
|
||||
"""抓取 BaoStock 日线行情。"""
|
||||
cfg = get_fetch_config()
|
||||
retry = max(int(cfg.get("retry", 3)), 3)
|
||||
delay = max(float(cfg.get("delay", 0.1)), 0.2)
|
||||
"""兼容旧调用签名；当前仅保留默认日线写入。"""
|
||||
if status_only:
|
||||
raise ValueError("交易状态请使用 --trade-status")
|
||||
if source and source.lower() != "tdx":
|
||||
raise ValueError(f"当前仅保留默认行情实现，不支持: {source}")
|
||||
return fetch_tdx_daily(start_date=start_date, end_date=end_date, symbol=symbol)
|
||||
|
||||
if end_date is None:
|
||||
end_date = datetime.now().strftime("%Y%m%d")
|
||||
if start_date is None:
|
||||
start_date = (datetime.now() - timedelta(days=30)).strftime("%Y%m%d")
|
||||
|
||||
codes = _ensure_stock_codes(retry=retry, delay=delay)
|
||||
if not codes:
|
||||
_logger.error("无股票列表，无法抓取日线数据")
|
||||
return
|
||||
|
||||
trading_days = get_trading_days(start_date, end_date)
|
||||
td_count = len(trading_days)
|
||||
_logger.info("[BaoStock] 交易日历: %s ~ %s 共 %d 个交易日", start_date, end_date, td_count)
|
||||
|
||||
if not trading_days:
|
||||
_logger.error("无交易日数据，无法抓取日线数据")
|
||||
return
|
||||
|
||||
no_data_dates = _get_no_data_dates(start_date, end_date)
|
||||
|
||||
ed_fmt = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
ed_date = datetime.strptime(ed_fmt, "%Y-%m-%d").date()
|
||||
session = get_session()
|
||||
try:
|
||||
not_listed = {row[0] for row in session.execute(
|
||||
select(StockInfo.code).where(StockInfo.ipo_date > ed_date)
|
||||
)}
|
||||
finally:
|
||||
session.close()
|
||||
codes = [c for c in codes if c not in not_listed]
|
||||
if not_listed:
|
||||
_logger.info("[BaoStock] %d 只股票未上市，跳过", len(not_listed))
|
||||
|
||||
codes = list(dict.fromkeys(codes))
|
||||
codes, unsupported_codes = _split_supported_daily_codes(codes)
|
||||
if unsupported_codes:
|
||||
_logger.info(
|
||||
"[BaoStock] 已跳过 %d 只非沪深A股代码: %s",
|
||||
len(unsupported_codes),
|
||||
_preview_codes(unsupported_codes),
|
||||
)
|
||||
|
||||
_logger.info("[BaoStock] 正在分析 %d 只股票的数据缺口...", len(codes))
|
||||
gaps = _analyze_gaps(codes, start_date, end_date, trading_days)
|
||||
|
||||
complete_count = len(codes) - len(gaps)
|
||||
if complete_count > 0:
|
||||
_logger.info("[BaoStock] %d 只股票数据已完整，跳过", complete_count)
|
||||
|
||||
total = len(gaps)
|
||||
if total == 0:
|
||||
_logger.info("[BaoStock] 所有股票数据已完整，无需抓取")
|
||||
return
|
||||
|
||||
if no_data_only:
|
||||
_logger.info("[BaoStock] 仅更新停牌/无数据记录，不写入 stock_daily")
|
||||
|
||||
_logger.info("正在抓取日线行情 [BaoStock] %s ~ %s，%d 只需更新", start_date, end_date, total)
|
||||
|
||||
success = 0
|
||||
fail = 0
|
||||
done = 0
|
||||
t_start = time.time()
|
||||
|
||||
tasks = list(gaps.items())
|
||||
for code, g in tasks:
|
||||
gap_start = g[0].replace("-", "")
|
||||
gap_end = g[-1].replace("-", "")
|
||||
t0 = time.time()
|
||||
try:
|
||||
rows = _fetch_one_stock_with_retry(
|
||||
code,
|
||||
gap_start,
|
||||
gap_end,
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
)
|
||||
expected_days = [
|
||||
d for d in trading_days
|
||||
if g[0] <= d <= g[-1] and d not in no_data_dates.get(code, set())
|
||||
]
|
||||
if rows:
|
||||
if no_data_only:
|
||||
row_days = {str(row["date"]) for row in rows}
|
||||
missing_days = [day for day in expected_days if day not in row_days]
|
||||
if missing_days:
|
||||
saved = _mark_no_data_days(code, missing_days)
|
||||
if saved:
|
||||
_logger.info(
|
||||
"[BaoStock] %s 标记 %d 个停牌/无数据日期：%s ~ %s",
|
||||
code,
|
||||
saved,
|
||||
missing_days[0],
|
||||
missing_days[-1],
|
||||
)
|
||||
else:
|
||||
row_days = {str(row["date"]) for row in rows}
|
||||
missing_days = [day for day in expected_days if day not in row_days]
|
||||
if missing_days:
|
||||
patched_rows = _fetch_missing_daily_rows(
|
||||
code,
|
||||
missing_days,
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
)
|
||||
if patched_rows:
|
||||
rows.extend(patched_rows)
|
||||
row_days.update(str(row["date"]) for row in patched_rows)
|
||||
still_missing = [day for day in expected_days if day not in row_days]
|
||||
if still_missing:
|
||||
_logger.warning(
|
||||
"[BaoStock] %s 仍缺失 %d 个交易日：%s ~ %s",
|
||||
code,
|
||||
len(still_missing),
|
||||
still_missing[0],
|
||||
still_missing[-1],
|
||||
)
|
||||
else:
|
||||
_logger.warning(
|
||||
"[BaoStock] %s 范围结果缺少 %d 个交易日，未能按天补齐：%s ~ %s",
|
||||
code,
|
||||
len(missing_days),
|
||||
missing_days[0],
|
||||
missing_days[-1],
|
||||
)
|
||||
batch_upsert(StockDaily, rows, ["code", "date"])
|
||||
success += 1
|
||||
else:
|
||||
fail += 1
|
||||
if no_data_only:
|
||||
saved = _mark_no_data_days(code, expected_days)
|
||||
if saved and expected_days:
|
||||
_logger.info(
|
||||
"[BaoStock] %s 标记 %d 个停牌/无数据日期：%s ~ %s",
|
||||
code,
|
||||
saved,
|
||||
expected_days[0],
|
||||
expected_days[-1],
|
||||
)
|
||||
_logger.warning("[BaoStock] %s 没有返回数据，区间 %s ~ %s", code, g[0], g[-1])
|
||||
except Exception as exc:
|
||||
fail += 1
|
||||
_logger.warning("[BaoStock] %s 获取失败: %s，区间 %s ~ %s", code, exc, g[0], g[-1])
|
||||
|
||||
time.sleep(delay + random.uniform(0, min(delay * 0.2, 0.5)))
|
||||
|
||||
done += 1
|
||||
if done % 100 == 0 or done == total:
|
||||
elapsed = time.time() - t_start
|
||||
avg = elapsed / done
|
||||
eta = avg * (total - done)
|
||||
_logger.info(
|
||||
"[%d/%d] %s 缺口:%s~%s 耗时:%.1fs 成功:%d 失败:%d 剩余:%.0fs",
|
||||
done, total, code, g[0], g[-1], time.time() - t0, success, fail, eta,
|
||||
)
|
||||
|
||||
total_time = time.time() - t_start
|
||||
_logger.info("日线行情抓取完成 [BaoStock] 成功:%d 失败:%d 总耗时:%.1fs", success, fail, total_time)
|
||||
__all__ = ["fetch_daily"]
|
||||
|
||||
@@ -1,179 +0,0 @@
|
||||
"""指数日线行情抓取 — 通达信本地缓存。"""
|
||||
|
||||
import time
|
||||
from datetime import datetime
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from src.db import IndexDaily, TradingDay, batch_upsert, get_session
|
||||
from src.fetchers.tdx_client import get_market_data as tdx_get_market_data
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import select
|
||||
|
||||
_logger = get_logger("index")
|
||||
|
||||
|
||||
INDICES = {
|
||||
"000001": ("SH", "上证指数"),
|
||||
"000300": ("SH", "沪深300"),
|
||||
"000905": ("SH", "中证500"),
|
||||
"000852": ("SH", "中证1000"),
|
||||
"000688": ("SH", "科创50"),
|
||||
"399001": ("SZ", "深证成指"),
|
||||
"399006": ("SZ", "创业板指"),
|
||||
"399005": ("SZ", "中小板指"),
|
||||
}
|
||||
|
||||
|
||||
def _code_to_tdx(code: str) -> str:
|
||||
market, _ = INDICES[code]
|
||||
return f"{code}.{market}"
|
||||
|
||||
|
||||
def _clean(val):
|
||||
if val is None:
|
||||
return None
|
||||
if isinstance(val, str) and val.strip() == "":
|
||||
return None
|
||||
return val
|
||||
|
||||
|
||||
def _tdx_market_data_to_rows(code: str, market_data: dict) -> list[dict]:
|
||||
if not market_data:
|
||||
return []
|
||||
|
||||
lower_map = {str(key).lower(): value for key, value in market_data.items()}
|
||||
base_frame = None
|
||||
for field in ("close", "open", "high", "low", "amount", "volume"):
|
||||
frame = lower_map.get(field)
|
||||
if hasattr(frame, "index") and len(frame.index) > 0:
|
||||
base_frame = frame
|
||||
break
|
||||
|
||||
if base_frame is None:
|
||||
return []
|
||||
|
||||
rows: list[dict] = []
|
||||
for ts in base_frame.index:
|
||||
row = {"code": code, "date": str(ts)[:10]}
|
||||
valid_value = False
|
||||
for field, column in (("open", "open"), ("high", "high"), ("low", "low"),
|
||||
("close", "close"), ("volume", "volume"), ("amount", "amount")):
|
||||
frame = lower_map.get(field)
|
||||
if frame is None or ts not in frame.index:
|
||||
continue
|
||||
cell = frame.loc[ts]
|
||||
if isinstance(cell, pd.Series):
|
||||
if code in cell.index:
|
||||
value = cell[code]
|
||||
else:
|
||||
value = cell.iloc[0]
|
||||
else:
|
||||
value = cell
|
||||
value = _clean(value)
|
||||
row[column] = value
|
||||
if value is not None:
|
||||
valid_value = True
|
||||
if valid_value:
|
||||
rows.append(row)
|
||||
return rows
|
||||
|
||||
|
||||
def _fetch_one_index(code: str, sd: str, ed: str) -> list[dict] | None:
|
||||
"""抓取单个指数的日线数据。"""
|
||||
tdx_code = _code_to_tdx(code)
|
||||
market_data = tdx_get_market_data(
|
||||
[tdx_code],
|
||||
period="1d",
|
||||
start_time=f"{sd.replace('-', '')}000000",
|
||||
end_time=f"{ed.replace('-', '')}235959",
|
||||
count=-1,
|
||||
dividend_type="none",
|
||||
fill_data=True,
|
||||
)
|
||||
rows = _tdx_market_data_to_rows(code, market_data)
|
||||
return rows if rows else None
|
||||
|
||||
|
||||
def _analyze_index_gaps(code: str, sd: str, ed: str) -> list[tuple[str, str]]:
|
||||
"""分析单个指数在 [sd, ed] 范围内的数据缺口。"""
|
||||
session = get_session()
|
||||
try:
|
||||
trading_days = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
).scalars().all()
|
||||
if not trading_days:
|
||||
return [(sd, ed)]
|
||||
|
||||
existing = set(session.execute(
|
||||
select(IndexDaily.date)
|
||||
.where(IndexDaily.code == code)
|
||||
.where(IndexDaily.date >= sd)
|
||||
.where(IndexDaily.date <= ed)
|
||||
).scalars().all())
|
||||
|
||||
missing = [d for d in trading_days if d not in existing]
|
||||
if not missing:
|
||||
return []
|
||||
|
||||
gaps = []
|
||||
gap_start = missing[0]
|
||||
gap_end = missing[0]
|
||||
for d in missing[1:]:
|
||||
if (d - gap_end).days <= 3:
|
||||
gap_end = d
|
||||
else:
|
||||
gaps.append((str(gap_start), str(gap_end)))
|
||||
gap_start = d
|
||||
gap_end = d
|
||||
gaps.append((str(gap_start), str(gap_end)))
|
||||
return gaps
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def fetch_index(start_date: str | None = None, end_date: str | None = None):
|
||||
"""抓取主要指数日线行情。"""
|
||||
if end_date is None:
|
||||
end_date = datetime.now().strftime("%Y%m%d")
|
||||
if start_date is None:
|
||||
start_date = "19901219"
|
||||
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
|
||||
gaps_map: dict[str, list[tuple[str, str]]] = {}
|
||||
for code in INDICES:
|
||||
gaps = _analyze_index_gaps(code, sd, ed)
|
||||
if gaps:
|
||||
gaps_map[code] = gaps
|
||||
|
||||
if not gaps_map:
|
||||
_logger.info("指数数据 %s ~ %s 已完整，跳过", sd, ed)
|
||||
return
|
||||
|
||||
_logger.info("正在抓取指数日线 %s ~ %s，需补缺 %d 个", sd, ed, len(gaps_map))
|
||||
|
||||
success = 0
|
||||
fail = 0
|
||||
t_start = time.time()
|
||||
for code, gaps in gaps_map.items():
|
||||
_, name = INDICES[code]
|
||||
total_rows = 0
|
||||
t0 = time.time()
|
||||
for gap_sd, gap_ed in gaps:
|
||||
rows = _fetch_one_index(code, gap_sd, gap_ed)
|
||||
if rows:
|
||||
batch_upsert(IndexDaily, rows, ["code", "date"])
|
||||
total_rows += len(rows)
|
||||
if total_rows:
|
||||
success += 1
|
||||
_logger.info("%s(%s): 补缺 %d 区间, %d 天, %.1fs", name, code, len(gaps), total_rows, time.time() - t0)
|
||||
else:
|
||||
fail += 1
|
||||
_logger.warning("%s(%s): 无数据", name, code)
|
||||
|
||||
_logger.info("指数数据抓取完成，成功:%d 失败:%d", success, fail)
|
||||
@@ -0,0 +1,120 @@
|
||||
"""常用指数日线增量抓取。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.db import IndexDaily, batch_upsert, get_session
|
||||
from src.fetchers.tdx_daily import (
|
||||
_bar_date,
|
||||
_fetch_index_bars_payload,
|
||||
_to_float,
|
||||
connected_session,
|
||||
)
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("index_daily")
|
||||
|
||||
# market: 0=深市，1=沪市。与指数行情页面保持一致。
|
||||
INDEX_SPECS = (
|
||||
("000001", 1, "上证指数"),
|
||||
("399001", 0, "深证成指"),
|
||||
("000300", 1, "沪深300"),
|
||||
("000905", 1, "中证500"),
|
||||
("000852", 1, "中证1000"),
|
||||
("399005", 0, "中小100"),
|
||||
("399006", 0, "创业板指"),
|
||||
)
|
||||
|
||||
|
||||
def _iso_to_yyyymmdd(value: str) -> str:
|
||||
return value.replace("-", "")
|
||||
|
||||
|
||||
def _default_end_date() -> str:
|
||||
return datetime.now().strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _load_latest_dates() -> dict[str, str]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(IndexDaily.code, func.max(IndexDaily.date)).group_by(IndexDaily.code)
|
||||
)
|
||||
return {str(code): str(day) for code, day in rows if day}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _incremental_start(latest_day: str | None) -> str:
|
||||
if not latest_day:
|
||||
return "19901219"
|
||||
# 多取一周，用前收盘价重算首日涨跌幅，同时安全覆盖源数据修订。
|
||||
day = datetime.strptime(latest_day, "%Y-%m-%d") - timedelta(days=7)
|
||||
return day.strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _index_bars_to_rows(code: str, bars: list[dict], start_date: str, end_date: str) -> list[dict]:
|
||||
parsed = []
|
||||
for bar in bars:
|
||||
day = _bar_date(bar)
|
||||
if not day:
|
||||
continue
|
||||
# pytdx 的指数成交量单位为“万股”，index_daily 历史数据和前端统一使用“股”。
|
||||
volume = _to_float(bar.get("vol", bar.get("volume")))
|
||||
values = {
|
||||
"code": code,
|
||||
"date": day,
|
||||
"open": _to_float(bar.get("open")),
|
||||
"high": _to_float(bar.get("high")),
|
||||
"low": _to_float(bar.get("low")),
|
||||
"close": _to_float(bar.get("close")),
|
||||
"volume": volume * 10_000 if volume is not None else None,
|
||||
"amount": _to_float(bar.get("amount")),
|
||||
}
|
||||
if all(values[field] is not None for field in ("open", "high", "low", "close")):
|
||||
parsed.append(values)
|
||||
|
||||
parsed = sorted({row["date"]: row for row in parsed}.values(), key=lambda row: row["date"])
|
||||
for index, row in enumerate(parsed):
|
||||
previous_close = parsed[index - 1]["close"] if index else None
|
||||
row["pct_change"] = (
|
||||
round((row["close"] - previous_close) / previous_close * 100, 4)
|
||||
if previous_close
|
||||
else None
|
||||
)
|
||||
|
||||
start_iso = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
end_iso = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
return [row for row in parsed if start_iso <= row["date"] <= end_iso]
|
||||
|
||||
|
||||
def fetch_index_daily(start_date: str | None = None, end_date: str | None = None) -> int:
|
||||
"""增量更新指数行情页使用的全部指数，返回写入行数。"""
|
||||
end_date = end_date or _default_end_date()
|
||||
latest_dates = _load_latest_dates()
|
||||
total = 0
|
||||
failures: list[str] = []
|
||||
|
||||
with connected_session() as api:
|
||||
for code, market, name in INDEX_SPECS:
|
||||
code_start = start_date or _incremental_start(latest_dates.get(code))
|
||||
try:
|
||||
bars = _fetch_index_bars_payload(api, market, code, code_start)
|
||||
rows = _index_bars_to_rows(code, bars, code_start, end_date)
|
||||
batch_upsert(IndexDaily, rows, ["code", "date"])
|
||||
total += len(rows)
|
||||
latest = rows[-1]["date"] if rows else "无新增"
|
||||
_logger.info("指数日线 %s(%s) 写入 %d 行，最新 %s", name, code, len(rows), latest)
|
||||
except Exception as exc:
|
||||
failures.append(code)
|
||||
_logger.warning("指数日线 %s(%s) 更新失败: %s", name, code, exc)
|
||||
|
||||
if failures and len(failures) == len(INDEX_SPECS):
|
||||
raise RuntimeError("全部指数日线更新失败: " + ", ".join(failures))
|
||||
return total
|
||||
|
||||
|
||||
__all__ = ["INDEX_SPECS", "fetch_index_daily"]
|
||||
@@ -0,0 +1,990 @@
|
||||
"""Automatic industry-chain classification from constituents and disclosed products."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from concurrent.futures import ThreadPoolExecutor, as_completed
|
||||
from datetime import datetime, timedelta
|
||||
from functools import lru_cache
|
||||
import hashlib
|
||||
import json
|
||||
import re
|
||||
import threading
|
||||
import time
|
||||
from typing import Any
|
||||
|
||||
import pandas as pd
|
||||
from bs4 import BeautifulSoup
|
||||
from sqlalchemy import select, text, update
|
||||
|
||||
from src.db import IndustryNode, StockDaily, StockF10Evidence, StockIndustry, StockInfo, batch_upsert, get_session
|
||||
from src.db.industry import deactivate_stock_industry_sources, replace_stock_industry_memberships
|
||||
from src.log import get_logger
|
||||
from src.industry.models import ChainSpec, INDUSTRY_CHAIN_SOURCE
|
||||
from src.industry.catalogs import (
|
||||
CHAIN_SPECS,
|
||||
SEMICONDUCTOR_DEFINITIONS,
|
||||
SEMICONDUCTOR_RULES,
|
||||
ROBOT_DEFINITIONS,
|
||||
ROBOT_RULES,
|
||||
SPACE_DEFINITIONS,
|
||||
SPACE_RULES,
|
||||
CPO_DEFINITIONS,
|
||||
CPO_RULES,
|
||||
LOW_ALTITUDE_DEFINITIONS,
|
||||
LOW_ALTITUDE_RULES,
|
||||
AIDC_DEFINITIONS,
|
||||
AIDC_RULES,
|
||||
MILITARY_ELECTRONICS_DEFINITIONS,
|
||||
MILITARY_ELECTRONICS_RULES,
|
||||
CONSUMER_ELECTRONICS_DEFINITIONS,
|
||||
CONSUMER_ELECTRONICS_RULES,
|
||||
SOFTWARE_CLOUD_DEFINITIONS,
|
||||
SOFTWARE_CLOUD_RULES,
|
||||
ELECTRONIC_COMPONENT_DEFINITIONS,
|
||||
ELECTRONIC_COMPONENT_RULES,
|
||||
COMMUNICATION_NETWORK_DEFINITIONS,
|
||||
COMMUNICATION_NETWORK_RULES,
|
||||
SOLID_STATE_BATTERY_DEFINITIONS,
|
||||
SOLID_STATE_BATTERY_RULES,
|
||||
COAL_CHEMICAL_DEFINITIONS,
|
||||
COAL_CHEMICAL_RULES,
|
||||
ENERGY_STORAGE_DEFINITIONS,
|
||||
ENERGY_STORAGE_RULES,
|
||||
PV_DEFINITIONS,
|
||||
PV_RULES,
|
||||
WIND_DEFINITIONS,
|
||||
WIND_RULES,
|
||||
HYDROGEN_DEFINITIONS,
|
||||
HYDROGEN_RULES,
|
||||
NEW_ENERGY_AUTO_DEFINITIONS,
|
||||
NEW_ENERGY_AUTO_RULES,
|
||||
PETROCHEMICAL_DEFINITIONS,
|
||||
PETROCHEMICAL_RULES,
|
||||
STEEL_DEFINITIONS,
|
||||
STEEL_RULES,
|
||||
NONFERROUS_DEFINITIONS,
|
||||
NONFERROUS_RULES,
|
||||
AUTO_AFTERMARKET_DEFINITIONS,
|
||||
AUTO_AFTERMARKET_RULES,
|
||||
INNOVATIVE_DRUG_DEFINITIONS,
|
||||
INNOVATIVE_DRUG_RULES,
|
||||
MEDICAL_DEVICE_DEFINITIONS,
|
||||
MEDICAL_DEVICE_RULES,
|
||||
CHEMICAL_PHARMA_DEFINITIONS,
|
||||
CHEMICAL_PHARMA_RULES,
|
||||
TCM_DEFINITIONS,
|
||||
TCM_RULES,
|
||||
MEDICAL_SERVICE_DEFINITIONS,
|
||||
MEDICAL_SERVICE_RULES,
|
||||
PHARMA_DISTRIBUTION_DEFINITIONS,
|
||||
PHARMA_DISTRIBUTION_RULES,
|
||||
FOOD_BEVERAGE_DEFINITIONS,
|
||||
FOOD_BEVERAGE_RULES,
|
||||
TOURISM_DEFINITIONS,
|
||||
TOURISM_RULES,
|
||||
RETAIL_DEFINITIONS,
|
||||
RETAIL_RULES,
|
||||
CROP_DEFINITIONS,
|
||||
CROP_RULES,
|
||||
LIVESTOCK_DEFINITIONS,
|
||||
LIVESTOCK_RULES,
|
||||
AQUACULTURE_DEFINITIONS,
|
||||
AQUACULTURE_RULES,
|
||||
PET_DEFINITIONS,
|
||||
PET_RULES,
|
||||
HOME_APPLIANCE_DEFINITIONS,
|
||||
HOME_APPLIANCE_RULES,
|
||||
TEXTILE_DEFINITIONS,
|
||||
TEXTILE_RULES,
|
||||
LIGHT_HOME_DEFINITIONS,
|
||||
LIGHT_HOME_RULES,
|
||||
INFRASTRUCTURE_DEFINITIONS,
|
||||
INFRASTRUCTURE_RULES,
|
||||
EDUCATION_DEFINITIONS,
|
||||
EDUCATION_RULES,
|
||||
HR_SERVICE_DEFINITIONS,
|
||||
HR_SERVICE_RULES,
|
||||
MEDIA_CONTENT_DEFINITIONS,
|
||||
MEDIA_CONTENT_RULES,
|
||||
TRANSPORT_LOGISTICS_DEFINITIONS,
|
||||
TRANSPORT_LOGISTICS_RULES,
|
||||
UTILITY_DEFINITIONS,
|
||||
UTILITY_RULES,
|
||||
ENVIRONMENT_DEFINITIONS,
|
||||
ENVIRONMENT_RULES,
|
||||
REAL_ESTATE_DEFINITIONS,
|
||||
REAL_ESTATE_RULES,
|
||||
BUILDING_MATERIAL_DEFINITIONS,
|
||||
BUILDING_MATERIAL_RULES,
|
||||
FINANCE_DEFINITIONS,
|
||||
FINANCE_RULES,
|
||||
BUSINESS_INDUSTRY_DEFINITIONS,
|
||||
BUSINESS_INDUSTRY_RULES,
|
||||
)
|
||||
from src.industry.classification import (
|
||||
_WEAK_BUSINESS_CLAUSE_MARKERS,
|
||||
_DOWNSTREAM_CONTEXT_MARKERS,
|
||||
_EQUIPMENT_SUFFIX_MARKERS,
|
||||
_PROFILE_ONLY_KEYWORDS,
|
||||
_business_keyword_count,
|
||||
classify_chain_profiles,
|
||||
classify_chain_profile,
|
||||
classify_semiconductor_profile,
|
||||
)
|
||||
from src.industry.evidence import (
|
||||
F10_PROFILE_MARKER,
|
||||
_normalize_f10_evidence_text,
|
||||
parse_ths_concept_evidence_html,
|
||||
_is_valid_f10_concept_name,
|
||||
parse_ths_concepts_html,
|
||||
_decode_ths_f10_html,
|
||||
parse_ths_operate_profile_html,
|
||||
_format_ths_operate_profile,
|
||||
_profile_texts,
|
||||
_split_f10_profile_evidence,
|
||||
)
|
||||
|
||||
F10_CHAIN_ROOT_SOURCE = "industry_chain_f10"
|
||||
F10_CHAIN_ROOT_CODE = "F10_CHAIN_ROOT"
|
||||
F10_CHAIN_ROOT_NAME = "同花顺 F10 产业链"
|
||||
|
||||
_logger = get_logger("industry.chain")
|
||||
|
||||
_F10_REQUEST_INTERVAL = 0.5
|
||||
_f10_rate_lock = threading.Lock()
|
||||
_f10_next_request_at = 0.0
|
||||
_f10_cooldown_until = 0.0
|
||||
_f10_limit_streak = 0
|
||||
|
||||
|
||||
def _node_rows(spec: ChainSpec, checked_at: datetime) -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"source": spec.source,
|
||||
"industry_code": code,
|
||||
"industry_name": name,
|
||||
"parent_code": parent_code,
|
||||
"industry_level": level,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
for code, name, parent_code, level in spec.definitions
|
||||
]
|
||||
|
||||
|
||||
def semiconductor_chain_nodes(*, now: datetime | None = None) -> list[dict[str, Any]]:
|
||||
return _node_rows(CHAIN_SPECS[0], now or datetime.now())
|
||||
|
||||
|
||||
@lru_cache(maxsize=128)
|
||||
def fetch_ths_board_constituents(board_code: str, board_kind: str) -> list[str]:
|
||||
"""Fetch the accessible high/low ranked pages and merge their stock codes."""
|
||||
import py_mini_racer
|
||||
import requests
|
||||
from akshare.stock_feature.stock_board_industry_ths import _get_file_content_ths
|
||||
|
||||
js_runtime = py_mini_racer.MiniRacer()
|
||||
js_runtime.eval(_get_file_content_ths("ths.js"))
|
||||
route = "thshy" if board_kind == "thshy" else "gn"
|
||||
|
||||
def fetch_page(page: int, order: str = "desc") -> tuple[str, int]:
|
||||
if page == 1 and order == "desc":
|
||||
url = f"https://q.10jqka.com.cn/{route}/detail/code/{board_code}/"
|
||||
else:
|
||||
url = f"https://q.10jqka.com.cn/{route}/detail/code/{board_code}/field/199112/order/{order}/page/{page}/ajax/1/"
|
||||
last_error: Exception | None = None
|
||||
for _ in range(3):
|
||||
verification = js_runtime.call("v")
|
||||
headers = {
|
||||
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36",
|
||||
"Cookie": f"v={verification}",
|
||||
"hexin-v": verification,
|
||||
"Referer": f"https://q.10jqka.com.cn/{route}/detail/code/{board_code}/",
|
||||
}
|
||||
try:
|
||||
response = requests.get(url, headers=headers, timeout=20)
|
||||
response.raise_for_status()
|
||||
response.encoding = "gbk"
|
||||
if "<table" not in response.text or "代码" not in response.text:
|
||||
raise ValueError(f"同花顺板块 {board_code} 第 {page} 页返回验证页")
|
||||
match = re.search(r'class="page_info">\s*\d+/(\d+)', response.text)
|
||||
return response.text, int(match.group(1)) if match else 1
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
time.sleep(0.5)
|
||||
assert last_error is not None
|
||||
raise last_error
|
||||
|
||||
first_html, page_count = fetch_page(1)
|
||||
html_pages = [first_html]
|
||||
accessible_pages = min(5, (page_count + 1) // 2)
|
||||
for page in range(2, accessible_pages + 1):
|
||||
time.sleep(0.1)
|
||||
html_pages.append(fetch_page(page)[0])
|
||||
for page in range(1, accessible_pages + 1):
|
||||
time.sleep(0.1)
|
||||
html_pages.append(fetch_page(page, "asc")[0])
|
||||
|
||||
codes: set[str] = set()
|
||||
for html in html_pages:
|
||||
for table in pd.read_html(StringIO(html)):
|
||||
if "代码" not in table.columns:
|
||||
continue
|
||||
for value in table["代码"]:
|
||||
code = str(value).split(".", 1)[0].strip().zfill(6)
|
||||
if re.fullmatch(r"\d{6}", code):
|
||||
codes.add(code)
|
||||
return sorted(codes)
|
||||
|
||||
|
||||
def fetch_ths_semiconductor_constituents() -> list[str]:
|
||||
return fetch_ths_board_constituents("881121", "thshy")
|
||||
|
||||
|
||||
def _wait_for_f10_request_slot() -> None:
|
||||
global _f10_next_request_at
|
||||
while True:
|
||||
with _f10_rate_lock:
|
||||
now = time.monotonic()
|
||||
allowed_at = max(_f10_next_request_at, _f10_cooldown_until)
|
||||
if now >= allowed_at:
|
||||
_f10_next_request_at = now + _F10_REQUEST_INTERVAL
|
||||
return
|
||||
wait_seconds = allowed_at - now
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
|
||||
def _register_f10_rate_limit() -> float:
|
||||
global _f10_cooldown_until, _f10_limit_streak
|
||||
with _f10_rate_lock:
|
||||
_f10_limit_streak += 1
|
||||
cooldown = min(300.0, 10.0 * (2 ** (_f10_limit_streak - 1)))
|
||||
_f10_cooldown_until = max(_f10_cooldown_until, time.monotonic() + cooldown)
|
||||
return cooldown
|
||||
|
||||
|
||||
def _clear_f10_rate_limit() -> None:
|
||||
global _f10_limit_streak
|
||||
with _f10_rate_lock:
|
||||
_f10_limit_streak = 0
|
||||
|
||||
|
||||
def _fetch_ths_stock_concept_html(stock_code: str) -> str:
|
||||
import requests
|
||||
|
||||
url = f"https://basic.10jqka.com.cn/{stock_code}/concept.html"
|
||||
for attempt in range(5):
|
||||
_wait_for_f10_request_slot()
|
||||
response = requests.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=20)
|
||||
if response.status_code not in (403, 429):
|
||||
response.raise_for_status()
|
||||
_clear_f10_rate_limit()
|
||||
return _decode_ths_f10_html(response.content)
|
||||
cooldown = _register_f10_rate_limit()
|
||||
_logger.warning("%s 同花顺F10限流，%.0f秒后重试 (%d/5)", stock_code, cooldown, attempt + 1)
|
||||
response.raise_for_status()
|
||||
return ""
|
||||
|
||||
|
||||
def fetch_ths_stock_concept_evidence(stock_code: str) -> str:
|
||||
return parse_ths_concept_evidence_html(_fetch_ths_stock_concept_html(stock_code))
|
||||
|
||||
|
||||
def fetch_ths_stock_concepts(stock_code: str) -> list[dict[str, str]]:
|
||||
return parse_ths_concepts_html(_fetch_ths_stock_concept_html(stock_code))
|
||||
|
||||
|
||||
def fetch_ths_stock_operate_evidence(stock_code: str) -> str:
|
||||
import requests
|
||||
|
||||
url = f"https://basic.10jqka.com.cn/{stock_code}/operate.html"
|
||||
for attempt in range(5):
|
||||
_wait_for_f10_request_slot()
|
||||
response = requests.get(url, headers={"User-Agent": "Mozilla/5.0"}, timeout=20)
|
||||
if response.status_code not in (403, 429):
|
||||
response.raise_for_status()
|
||||
_clear_f10_rate_limit()
|
||||
html = _decode_ths_f10_html(response.content)
|
||||
profile = parse_ths_operate_profile_html(html)
|
||||
if not any(profile.get(field) for field in ("主营业务", "产品类型", "产品名称")):
|
||||
raise ValueError(f"{stock_code} 同花顺主营页无有效业务字段")
|
||||
return _format_ths_operate_profile(profile)
|
||||
cooldown = _register_f10_rate_limit()
|
||||
_logger.warning("%s 同花顺F10主营限流，%.0f秒后重试 (%d/5)", stock_code, cooldown, attempt + 1)
|
||||
response.raise_for_status()
|
||||
return ""
|
||||
|
||||
|
||||
def fetch_ths_stock_industry_evidence(stock_code: str) -> str:
|
||||
"""主营页缺失时，从同花顺公司概况读取申万行业作为自动补充证据。"""
|
||||
import requests
|
||||
|
||||
_wait_for_f10_request_slot()
|
||||
response = requests.get(
|
||||
f"https://basic.10jqka.com.cn/{stock_code}/company.html",
|
||||
headers={"User-Agent": "Mozilla/5.0"}, timeout=20,
|
||||
)
|
||||
response.raise_for_status()
|
||||
soup = BeautifulSoup(_decode_ths_f10_html(response.content), "html.parser")
|
||||
label = soup.find("strong", string=re.compile(r"所属申万行业"))
|
||||
value = label.find_next("span").get_text(" ", strip=True) if label else ""
|
||||
return _format_ths_operate_profile({"主营业务": f"同花顺所属申万行业：{value}"}) if value else ""
|
||||
|
||||
|
||||
def fetch_ths_stock_chain_evidence(stock_code: str) -> str:
|
||||
concept_evidence = fetch_ths_stock_concept_evidence(stock_code)
|
||||
operate_evidence = fetch_ths_stock_operate_evidence(stock_code)
|
||||
return _normalize_f10_evidence_text(" ".join(part for part in (concept_evidence, operate_evidence) if part))
|
||||
|
||||
|
||||
def _refresh_ths_stock_chain_evidence(stock_code: str, existing_evidence: str = "") -> str:
|
||||
if existing_evidence and F10_PROFILE_MARKER not in existing_evidence:
|
||||
operate_evidence = fetch_ths_stock_operate_evidence(stock_code)
|
||||
return _normalize_f10_evidence_text(" ".join(part for part in (existing_evidence, operate_evidence) if part))
|
||||
return fetch_ths_stock_chain_evidence(stock_code)
|
||||
|
||||
|
||||
_NON_INDUSTRY_CONCEPT_FRAGMENTS = (
|
||||
"融资融券", "转融券", "沪股通", "深股通", "陆股通", "标普道琼斯", "富时罗素", "msci",
|
||||
"证金持股", "社保重仓", "基金重仓", "机构重仓", "养老金持股", "同花顺漂亮", "同花顺中特估",
|
||||
"昨日涨停", "昨日连板", "预盈预增", "高送转", "送转填权", "摘帽", "st板块",
|
||||
)
|
||||
|
||||
|
||||
def _is_limit_up_stock(stock_code: str, stock_name: str, pct_change: float) -> bool:
|
||||
name = (stock_name or "").upper()
|
||||
code = str(stock_code)
|
||||
if "ST" in name:
|
||||
return pct_change >= 4.8
|
||||
if code.startswith(("300", "301", "688")):
|
||||
return pct_change >= 19.5
|
||||
if code.startswith(("4", "8", "92")):
|
||||
return pct_change >= 29.5
|
||||
return pct_change >= 9.5
|
||||
|
||||
|
||||
def _latest_limit_up_codes(symbol: str | None = None, limit: int | None = None) -> tuple[Any, list[str]]:
|
||||
session = get_session()
|
||||
try:
|
||||
latest_date = session.execute(select(StockDaily.date).order_by(StockDaily.date.desc()).limit(1)).scalar_one_or_none()
|
||||
if latest_date is None:
|
||||
return None, []
|
||||
rows = session.execute(
|
||||
select(StockDaily.code, StockDaily.pct_change, StockInfo.name)
|
||||
.outerjoin(StockInfo, StockInfo.code == StockDaily.code)
|
||||
.where(StockDaily.date == latest_date, StockDaily.turnover > 0)
|
||||
)
|
||||
codes = sorted(
|
||||
str(code)
|
||||
for code, pct_change, name in rows
|
||||
if _is_limit_up_stock(str(code), str(name or ""), float(pct_change or 0))
|
||||
)
|
||||
finally:
|
||||
session.close()
|
||||
if symbol:
|
||||
requested = {
|
||||
part.strip().upper().replace("SH", "").replace("SZ", "").replace("BJ", "")
|
||||
for part in symbol.split(",") if part.strip()
|
||||
}
|
||||
codes = [code for code in codes if code in requested]
|
||||
if limit is not None:
|
||||
codes = codes[:max(int(limit), 0)]
|
||||
return latest_date, codes
|
||||
|
||||
|
||||
def _load_f10_concept_cache(max_age_hours: int = 24) -> dict[str, list[dict[str, str]]]:
|
||||
cutoff = datetime.now() - timedelta(hours=max_age_hours)
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(StockF10Evidence.stock_code, StockF10Evidence.evidence_text)
|
||||
.where(StockF10Evidence.fetched_at >= cutoff)
|
||||
)
|
||||
result: dict[str, list[dict[str, str]]] = {}
|
||||
for stock_code, payload in rows:
|
||||
try:
|
||||
concepts = json.loads(str(payload))
|
||||
except (TypeError, ValueError, json.JSONDecodeError):
|
||||
continue
|
||||
if isinstance(concepts, list):
|
||||
result[str(stock_code)] = [
|
||||
{"name": str(item.get("name") or "").strip(), "explanation": str(item.get("explanation") or "").strip()}
|
||||
for item in concepts if isinstance(item, dict) and str(item.get("name") or "").strip()
|
||||
]
|
||||
return result
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _f10_tree_identity(concept_name: str) -> tuple[str, str]:
|
||||
digest = hashlib.sha1(concept_name.encode("utf-8")).hexdigest()
|
||||
return f"industry_chain_{digest[:5]}", f"F10_{digest[:12].upper()}"
|
||||
|
||||
|
||||
F10_SEMICONDUCTOR_CHAIN_CODE = "F10_CHAIN_SEMI"
|
||||
F10_SEMICONDUCTOR_CHAIN_NAME = "半导体产业链"
|
||||
F10_SEMICONDUCTOR_STAGES = (
|
||||
("原材料", ("半导体材料", "电子特气", "光刻胶", "硅片", "晶圆材料", "靶材", "抛光液", "光掩模", "封装材料", "引线框架", "湿电子化学品")),
|
||||
("设备", ("半导体设备", "半导体装备", "集成电路设备", "电子工艺装备", "刻蚀机", "薄膜沉积", "涂胶显影", "离子注入", "清洗设备", "检测设备", "测试机", "探针台", "光刻机")),
|
||||
("EDA/IP", ("eda", "ip授权", "ip核", "知识产权核", "集成电路设计软件")),
|
||||
("设计", ("芯片设计", "集成电路设计", "芯片研发", "集成电路研发", "存储器", "存储芯片", "nor flash", "nand", "dram", "eeprom", "模拟芯片", "模拟集成电路", "电源管理芯片", "信号链", "功率芯片", "功率半导体", "igbt", "mosfet", "传感器芯片", "mems", "光芯片", "射频芯片", "图像传感器", "数字芯片", "微控制器", "mcu", "cpu", "gpu", "soc", "fpga", "asic")),
|
||||
("制造", ("晶圆代工", "晶圆制造", "集成电路晶圆", "芯片制造")),
|
||||
("封测", ("封装测试", "封装、测试", "集成电路封测", "封测服务", "晶圆测试")),
|
||||
("IDM/分立器件", ("功率器件制造", "分立器件制造", "半导体分立器件", "idm")),
|
||||
("下游应用", ("汽车电子系统", "车载终端", "智能座舱系统", "消费电子整机", "智能终端整机", "可穿戴设备", "通信终端", "物联网终端", "通信系统设备", "工业控制系统", "新能源控制系统", "电力电子系统", "人工智能服务器", "数据中心设备", "ai服务器")),
|
||||
("综合", ()),
|
||||
)
|
||||
_SEMICONDUCTOR_CONCEPT_TERMS = ("半导体", "芯片", "集成电路", "晶圆", "光刻", "封测")
|
||||
|
||||
|
||||
def _is_semiconductor_concept_name(name: str) -> bool:
|
||||
normalized = str(name or "").lower()
|
||||
return any(term in normalized for term in _SEMICONDUCTOR_CONCEPT_TERMS)
|
||||
|
||||
|
||||
def _semiconductor_stage_names(concepts: list[dict[str, str]]) -> list[str]:
|
||||
text_value = " ".join(
|
||||
f"{concept.get('name', '')} {concept.get('explanation', '')}" for concept in concepts
|
||||
).lower()
|
||||
matched = [
|
||||
stage_name
|
||||
for stage_name, keywords in F10_SEMICONDUCTOR_STAGES
|
||||
if keywords and any(keyword in text_value for keyword in keywords)
|
||||
]
|
||||
if not matched and any(_is_semiconductor_concept_name(concept.get("name", "")) for concept in concepts):
|
||||
matched.append("综合")
|
||||
return matched
|
||||
|
||||
|
||||
def _industry_concept_names(concepts: list[dict[str, str]]) -> list[str]:
|
||||
names = []
|
||||
for concept in concepts:
|
||||
name = str(concept.get("name") or "").strip()
|
||||
normalized = name.lower()
|
||||
if (
|
||||
not _is_valid_f10_concept_name(name)
|
||||
or any(fragment in normalized for fragment in _NON_INDUSTRY_CONCEPT_FRAGMENTS)
|
||||
or _is_semiconductor_concept_name(name)
|
||||
):
|
||||
continue
|
||||
names.append(name)
|
||||
return list(dict.fromkeys(names))
|
||||
|
||||
|
||||
def _deactivate_node_ids(session, node_ids: list[int], checked_at: datetime) -> None:
|
||||
for offset in range(0, len(node_ids), 500):
|
||||
session.execute(
|
||||
update(IndustryNode)
|
||||
.where(IndustryNode.id.in_(node_ids[offset:offset + 500]))
|
||||
.values(active=False, updated_at=checked_at)
|
||||
)
|
||||
|
||||
|
||||
def _deactivate_existing_industry_classifications(checked_at: datetime) -> None:
|
||||
session = get_session()
|
||||
try:
|
||||
node_ids = list(session.scalars(select(IndustryNode.id).where(IndustryNode.source.like("industry_chain%"), IndustryNode.active.is_(True))))
|
||||
_deactivate_node_ids(session, node_ids, checked_at)
|
||||
sources = list(session.scalars(
|
||||
select(StockIndustry.source)
|
||||
.where(StockIndustry.source.like("industry_chain%"), StockIndustry.active.is_(True))
|
||||
.distinct()
|
||||
))
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
deactivate_stock_industry_sources(sources)
|
||||
|
||||
|
||||
def _sync_limit_up_f10_tree(
|
||||
*, symbol: str | None, limit: int | None, refresh: bool, workers: int,
|
||||
) -> tuple[int, int, int]:
|
||||
effective_date, candidates = _latest_limit_up_codes(symbol=symbol, limit=limit)
|
||||
if not candidates:
|
||||
return 0, 0, 0
|
||||
|
||||
concept_cache = {} if refresh else _load_f10_concept_cache()
|
||||
fetch_codes = [code for code in candidates if code not in concept_cache]
|
||||
failed: set[str] = set()
|
||||
cache_rows: list[dict[str, Any]] = []
|
||||
with ThreadPoolExecutor(max_workers=max(1, min(int(workers), 8))) as executor:
|
||||
futures = {executor.submit(fetch_ths_stock_concepts, code): code for code in fetch_codes if not code.startswith("92")}
|
||||
failed.update(code for code in fetch_codes if code.startswith("92"))
|
||||
for completed, future in enumerate(as_completed(futures), start=1):
|
||||
stock_code = futures[future]
|
||||
try:
|
||||
concept_cache[stock_code] = future.result()
|
||||
cache_rows.append({
|
||||
"stock_code": stock_code,
|
||||
"evidence_text": json.dumps(concept_cache[stock_code], ensure_ascii=False),
|
||||
"fetched_at": datetime.now(),
|
||||
})
|
||||
if len(cache_rows) >= 25:
|
||||
batch_upsert(StockF10Evidence, cache_rows, ["stock_code"])
|
||||
cache_rows.clear()
|
||||
except Exception as exc:
|
||||
failed.add(stock_code)
|
||||
_logger.warning("%s 同花顺F10分类获取失败: %s", stock_code, exc)
|
||||
if completed % 25 == 0 or completed == len(futures):
|
||||
_logger.info("涨停股F10抓取 %d/%d，失败:%d", completed, len(futures), len(failed))
|
||||
if cache_rows:
|
||||
batch_upsert(StockF10Evidence, cache_rows, ["stock_code"])
|
||||
|
||||
checked_at = datetime.now()
|
||||
if symbol is None and limit is None:
|
||||
_deactivate_existing_industry_classifications(checked_at)
|
||||
|
||||
node_rows: dict[tuple[str, str], dict[str, Any]] = {}
|
||||
node_rows[(F10_CHAIN_ROOT_SOURCE, F10_CHAIN_ROOT_CODE)] = {
|
||||
"source": F10_CHAIN_ROOT_SOURCE,
|
||||
"industry_code": F10_CHAIN_ROOT_CODE,
|
||||
"industry_name": F10_CHAIN_ROOT_NAME,
|
||||
"parent_code": "",
|
||||
"industry_level": 1,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
node_rows[(F10_CHAIN_ROOT_SOURCE, F10_SEMICONDUCTOR_CHAIN_CODE)] = {
|
||||
"source": F10_CHAIN_ROOT_SOURCE,
|
||||
"industry_code": F10_SEMICONDUCTOR_CHAIN_CODE,
|
||||
"industry_name": F10_SEMICONDUCTOR_CHAIN_NAME,
|
||||
"parent_code": F10_CHAIN_ROOT_CODE,
|
||||
"industry_level": 2,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
semiconductor_stage_nodes: dict[str, tuple[str, str]] = {}
|
||||
for stage_name, _ in F10_SEMICONDUCTOR_STAGES:
|
||||
source, industry_code = _f10_tree_identity(f"{F10_SEMICONDUCTOR_CHAIN_NAME}/{stage_name}")
|
||||
semiconductor_stage_nodes[stage_name] = (source, industry_code)
|
||||
node_rows[(source, industry_code)] = {
|
||||
"source": source,
|
||||
"industry_code": industry_code,
|
||||
"industry_name": stage_name,
|
||||
"parent_code": F10_SEMICONDUCTOR_CHAIN_CODE,
|
||||
"industry_level": 3,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
membership_rows: list[dict[str, Any]] = []
|
||||
for stock_code in candidates:
|
||||
if stock_code in failed:
|
||||
continue
|
||||
stock_concepts = concept_cache.get(stock_code, [])
|
||||
semiconductor_stages = _semiconductor_stage_names(stock_concepts)
|
||||
for stage_name in semiconductor_stages:
|
||||
source, industry_code = semiconductor_stage_nodes[stage_name]
|
||||
membership_rows.append({
|
||||
"source": source,
|
||||
"stock_code": stock_code,
|
||||
"industry_code": industry_code,
|
||||
"effective_date": effective_date,
|
||||
"updated_at": checked_at,
|
||||
})
|
||||
|
||||
concept_names = _industry_concept_names(stock_concepts)
|
||||
if not concept_names and not semiconductor_stages:
|
||||
concept_names = ["F10暂无分类"]
|
||||
for concept_name in concept_names:
|
||||
source, industry_code = _f10_tree_identity(concept_name)
|
||||
node_rows[(source, industry_code)] = {
|
||||
"source": source,
|
||||
"industry_code": industry_code,
|
||||
"industry_name": concept_name,
|
||||
"parent_code": F10_CHAIN_ROOT_CODE,
|
||||
"industry_level": 2,
|
||||
"active": True,
|
||||
"updated_at": checked_at,
|
||||
}
|
||||
membership_rows.append({
|
||||
"source": source,
|
||||
"stock_code": stock_code,
|
||||
"industry_code": industry_code,
|
||||
"effective_date": effective_date,
|
||||
"updated_at": checked_at,
|
||||
})
|
||||
batch_upsert(IndustryNode, list(node_rows.values()), ["source", "industry_code"])
|
||||
replace_stock_industry_memberships(membership_rows)
|
||||
_logger.info(
|
||||
"涨停股F10分类完成，交易日:%s，涨停:%d，分类:%d，归属:%d，F10失败:%d",
|
||||
effective_date, len(candidates), len(node_rows), len(membership_rows), len(failed),
|
||||
)
|
||||
return len(node_rows), len(membership_rows), len(failed)
|
||||
|
||||
|
||||
def _existing_chain_codes(source: str) -> set[str]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(select(StockIndustry.stock_code).where(StockIndustry.source == source, StockIndustry.active.is_(True)))
|
||||
return {str(row[0]) for row in rows}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _load_f10_evidence_cache(max_age_hours: int = 24) -> dict[str, str]:
|
||||
cutoff = datetime.now() - timedelta(hours=max_age_hours)
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(StockF10Evidence.stock_code, StockF10Evidence.evidence_text)
|
||||
.where(StockF10Evidence.fetched_at >= cutoff)
|
||||
)
|
||||
result: dict[str, str] = {}
|
||||
for stock_code, evidence_text in rows:
|
||||
payload = str(evidence_text)
|
||||
try:
|
||||
concepts = json.loads(payload)
|
||||
except (TypeError, ValueError, json.JSONDecodeError):
|
||||
result[str(stock_code)] = _normalize_f10_evidence_text(payload)
|
||||
continue
|
||||
if isinstance(concepts, list):
|
||||
result[str(stock_code)] = _normalize_f10_evidence_text(" ".join(
|
||||
str(item.get("explanation") or "").strip()
|
||||
for item in concepts
|
||||
if isinstance(item, dict) and str(item.get("explanation") or "").strip()
|
||||
))
|
||||
else:
|
||||
result[str(stock_code)] = _normalize_f10_evidence_text(payload)
|
||||
return result
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _activate_nodes(spec: ChainSpec, checked_at: datetime) -> int:
|
||||
session = get_session()
|
||||
try:
|
||||
node_ids = list(session.scalars(select(IndustryNode.id).where(IndustryNode.source == spec.source, IndustryNode.active.is_(True))))
|
||||
_deactivate_node_ids(session, node_ids, checked_at)
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
rows = _node_rows(spec, checked_at)
|
||||
batch_upsert(IndustryNode, rows, ["source", "industry_code"])
|
||||
return len(rows)
|
||||
|
||||
|
||||
def _sync_chain(
|
||||
spec: ChainSpec,
|
||||
*,
|
||||
symbol: str | None,
|
||||
limit: int | None,
|
||||
refresh: bool,
|
||||
workers: int,
|
||||
evidence_cache: dict[str, str],
|
||||
) -> tuple[int, int, int]:
|
||||
requested = {
|
||||
part.strip().upper().replace("SH", "").replace("SZ", "").replace("BJ", "")
|
||||
for part in (symbol or "").split(",")
|
||||
if part.strip()
|
||||
}
|
||||
boards = ((spec.board_kind, spec.board_code), *spec.additional_boards)
|
||||
board_codes: set[str] = set()
|
||||
board_failures = 0
|
||||
for board_kind, board_code in (() if requested else boards):
|
||||
try:
|
||||
board_codes.update(fetch_ths_board_constituents(board_code, board_kind))
|
||||
except Exception as exc:
|
||||
board_failures += 1
|
||||
_logger.warning("%s 候选板块 %s/%s 获取失败，使用已有归属与F10缓存继续分类: %s", spec.key, board_kind, board_code, exc)
|
||||
existing = _existing_chain_codes(spec.source)
|
||||
cached_matches = {
|
||||
stock_code
|
||||
for stock_code, evidence_text in evidence_cache.items()
|
||||
if classify_chain_profiles(None, spec, evidence_text, allow_default=False)
|
||||
}
|
||||
candidates = sorted(board_codes | existing | cached_matches)
|
||||
if board_failures:
|
||||
_logger.info("%s 候选板块成功:%d，失败:%d，缓存命中:%d", spec.key, len(boards) - board_failures, board_failures, len(cached_matches))
|
||||
if requested:
|
||||
# 显式指定的股票必须进入候选池；最终是否归属仍完全由 F10 业务证据决定。
|
||||
candidates = sorted(requested)
|
||||
if limit is not None:
|
||||
candidates = candidates[: max(int(limit), 0)]
|
||||
if not candidates:
|
||||
return 0, 0, 0
|
||||
|
||||
unsupported_codes = {code for code in candidates if code.startswith("92") and code not in evidence_cache}
|
||||
errors: set[str] = set(unsupported_codes)
|
||||
if unsupported_codes:
|
||||
_logger.info("%s 跳过同花顺F10暂不支持的920代码:%d", spec.key, len(unsupported_codes))
|
||||
evidence_candidates = [
|
||||
code for code in candidates
|
||||
if code not in unsupported_codes
|
||||
and (code not in evidence_cache or (refresh and F10_PROFILE_MARKER not in evidence_cache[code]))
|
||||
]
|
||||
evidence_rows: list[dict[str, Any]] = []
|
||||
with ThreadPoolExecutor(max_workers=max(1, min(int(workers), 8))) as executor:
|
||||
futures = {
|
||||
executor.submit(_refresh_ths_stock_chain_evidence, code, evidence_cache.get(code, "")): code
|
||||
for code in evidence_candidates
|
||||
}
|
||||
completed = 0
|
||||
for future in as_completed(futures):
|
||||
stock_code = futures[future]
|
||||
completed += 1
|
||||
try:
|
||||
evidence_cache[stock_code] = future.result()
|
||||
evidence_rows.append({
|
||||
"stock_code": stock_code,
|
||||
"evidence_text": evidence_cache[stock_code],
|
||||
"fetched_at": datetime.now(),
|
||||
})
|
||||
if len(evidence_rows) >= 50:
|
||||
batch_upsert(StockF10Evidence, evidence_rows, ["stock_code"])
|
||||
evidence_rows.clear()
|
||||
except Exception as exc:
|
||||
errors.add(stock_code)
|
||||
_logger.warning("%s %s同花顺F10获取失败: %s", spec.key, stock_code, exc)
|
||||
if completed % 50 == 0 or completed == len(evidence_candidates):
|
||||
_logger.info("%s F10抓取 %d/%d，失败:%d", spec.key, completed, len(evidence_candidates), len(errors))
|
||||
if evidence_rows:
|
||||
batch_upsert(StockF10Evidence, evidence_rows, ["stock_code"])
|
||||
|
||||
checked_at = datetime.now()
|
||||
rows: list[dict[str, Any]] = []
|
||||
matched = 0
|
||||
unmatched = 0
|
||||
unmatched_code = f"{spec.definitions[0][0]}_UNMATCHED"
|
||||
for stock_code in candidates:
|
||||
if stock_code in errors or stock_code not in evidence_cache:
|
||||
continue
|
||||
allow_default = not requested and stock_code in (board_codes | existing)
|
||||
industry_codes = classify_chain_profiles(
|
||||
None,
|
||||
spec,
|
||||
evidence_cache[stock_code],
|
||||
allow_default=allow_default,
|
||||
)
|
||||
if industry_codes:
|
||||
matched += 1
|
||||
else:
|
||||
unmatched += 1
|
||||
for industry_code in industry_codes or [unmatched_code]:
|
||||
rows.append({
|
||||
"source": spec.source,
|
||||
"stock_code": stock_code,
|
||||
"industry_code": industry_code,
|
||||
"effective_date": None,
|
||||
"updated_at": checked_at,
|
||||
})
|
||||
if rows:
|
||||
replace_stock_industry_memberships(rows)
|
||||
_logger.info("%s 产业链分类完成，候选:%d，匹配:%d，排除:%d，归属:%d，失败:%d", spec.key, len(candidates), matched, unmatched, len(rows), len(errors))
|
||||
return matched, unmatched, len(errors)
|
||||
|
||||
|
||||
def sync_industry_chains(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
taxonomy_only: bool = False,
|
||||
refresh: bool = False,
|
||||
workers: int = 8,
|
||||
) -> tuple[int, int, int]:
|
||||
checked_at = datetime.now()
|
||||
curated_sources = [spec.source for spec in CHAIN_SPECS]
|
||||
session = get_session()
|
||||
try:
|
||||
obsolete_sources = list(session.scalars(
|
||||
select(StockIndustry.source)
|
||||
.where(StockIndustry.source.like("industry_chain%"), StockIndustry.source.notin_(curated_sources), StockIndustry.active.is_(True))
|
||||
.distinct()
|
||||
))
|
||||
node_ids = list(session.scalars(
|
||||
select(IndustryNode.id)
|
||||
.where(IndustryNode.source.like("industry_chain%"), IndustryNode.source.notin_(curated_sources), IndustryNode.active.is_(True))
|
||||
))
|
||||
_deactivate_node_ids(session, node_ids, checked_at)
|
||||
session.commit()
|
||||
finally:
|
||||
session.close()
|
||||
deactivate_stock_industry_sources(obsolete_sources)
|
||||
|
||||
node_count = sum(_activate_nodes(spec, checked_at) for spec in CHAIN_SPECS)
|
||||
if taxonomy_only:
|
||||
return node_count, 0, 0
|
||||
|
||||
evidence_cache = {} if refresh else _load_f10_evidence_cache()
|
||||
matched = 0
|
||||
failed = 0
|
||||
for spec in CHAIN_SPECS:
|
||||
chain_matched, _, chain_failed = _sync_chain(
|
||||
spec,
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
refresh=refresh,
|
||||
workers=workers,
|
||||
evidence_cache=evidence_cache,
|
||||
)
|
||||
matched += chain_matched
|
||||
failed += chain_failed
|
||||
return node_count, matched, failed
|
||||
|
||||
|
||||
def _unclassified_active_candidates(end_date: str | None = None) -> list[dict[str, Any]]:
|
||||
"""返回符合情绪活跃条件、但尚无有效产业链归属的股票。"""
|
||||
session = get_session()
|
||||
try:
|
||||
resolved_end = end_date or session.execute(select(StockDaily.date).order_by(StockDaily.date.desc()).limit(1)).scalar()
|
||||
if not resolved_end:
|
||||
return []
|
||||
rows = session.execute(text("""
|
||||
WITH recent_dates AS (
|
||||
SELECT date FROM trading_day WHERE date <= :end_date ORDER BY date DESC LIMIT 35
|
||||
), date_bounds AS (
|
||||
SELECT MIN(date) AS history_start,
|
||||
(SELECT date FROM recent_dates ORDER BY date DESC LIMIT 1 OFFSET 29) AS candidate_start
|
||||
FROM recent_dates
|
||||
), quotes AS (
|
||||
SELECT sd.code, sd.date, sd.close, sd.pct_change, sd.turnover, info.name,
|
||||
LAG(sd.close, 5) OVER (PARTITION BY sd.code ORDER BY sd.date) AS prev5
|
||||
FROM stock_daily sd
|
||||
LEFT JOIN stock_info info ON info.code = sd.code
|
||||
CROSS JOIN date_bounds bounds
|
||||
WHERE sd.date >= bounds.history_start AND sd.date <= :end_date AND sd.turnover > 0
|
||||
), candidates AS (
|
||||
SELECT q.code, MAX(q.name) AS name,
|
||||
MAX(CASE WHEN q.date = :end_date THEN q.turnover ELSE 0 END) AS turnover
|
||||
FROM quotes q CROSS JOIN date_bounds bounds
|
||||
WHERE q.date >= bounds.candidate_start
|
||||
GROUP BY q.code
|
||||
HAVING MAX(CASE WHEN
|
||||
q.pct_change >= CASE
|
||||
WHEN UPPER(q.name) LIKE '%ST%' THEN 4.8
|
||||
WHEN q.code LIKE '300%' OR q.code LIKE '301%' OR q.code LIKE '688%' THEN 19.5
|
||||
WHEN q.code LIKE '4%' OR q.code LIKE '8%' OR q.code LIKE '92%' THEN 29.5
|
||||
ELSE 9.5 END
|
||||
OR (q.prev5 > 0 AND (q.close / q.prev5 - 1) * 100 >= CASE
|
||||
WHEN UPPER(q.name) LIKE '%ST%' THEN 4.8
|
||||
WHEN q.code LIKE '300%' OR q.code LIKE '301%' OR q.code LIKE '688%' THEN 19.5
|
||||
WHEN q.code LIKE '4%' OR q.code LIKE '8%' OR q.code LIKE '92%' THEN 29.5
|
||||
ELSE 9.5 END)
|
||||
THEN 1 ELSE 0 END) = 1
|
||||
), active_map AS (
|
||||
SELECT DISTINCT membership.stock_code
|
||||
FROM stock_industry membership
|
||||
INNER JOIN industry_node node
|
||||
ON node.source = membership.source
|
||||
AND node.industry_code = membership.industry_code
|
||||
AND node.active = 1
|
||||
WHERE membership.source LIKE 'industry_chain%' AND membership.active = 1
|
||||
)
|
||||
SELECT candidates.code, candidates.name, candidates.turnover
|
||||
FROM candidates
|
||||
LEFT JOIN active_map ON active_map.stock_code = candidates.code
|
||||
WHERE active_map.stock_code IS NULL AND candidates.turnover > 0
|
||||
ORDER BY candidates.turnover DESC, candidates.code
|
||||
"""), {"end_date": str(resolved_end)[:10]}).mappings()
|
||||
return [dict(row) for row in rows]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def classify_unclassified_active(
|
||||
*, end_date: str | None = None, refresh: bool = False, limit: int | None = None, workers: int = 8,
|
||||
) -> tuple[int, int, int]:
|
||||
"""批量用 F10 主营将情绪活跃但未入链的股票匹配到全部产业链。"""
|
||||
candidates = _unclassified_active_candidates(end_date)
|
||||
if limit is not None:
|
||||
candidates = candidates[:max(int(limit), 0)]
|
||||
codes = [str(item["code"]) for item in candidates]
|
||||
if not codes:
|
||||
return 0, 0, 0
|
||||
|
||||
checked_at = datetime.now()
|
||||
for spec in CHAIN_SPECS:
|
||||
_activate_nodes(spec, checked_at)
|
||||
evidence_cache = _load_f10_evidence_cache()
|
||||
fetch_codes = [
|
||||
code for code in codes
|
||||
if not code.startswith("92")
|
||||
and (refresh or code not in evidence_cache or F10_PROFILE_MARKER not in evidence_cache[code])
|
||||
]
|
||||
failed: set[str] = set(code for code in codes if code.startswith("92") and code not in evidence_cache)
|
||||
evidence_rows: list[dict[str, Any]] = []
|
||||
with ThreadPoolExecutor(max_workers=max(1, min(int(workers), 8))) as executor:
|
||||
futures = {
|
||||
executor.submit(_refresh_ths_stock_chain_evidence, code, evidence_cache.get(code, "")): code
|
||||
for code in fetch_codes
|
||||
}
|
||||
for completed, future in enumerate(as_completed(futures), start=1):
|
||||
code = futures[future]
|
||||
try:
|
||||
evidence_cache[code] = future.result()
|
||||
evidence_rows.append({"stock_code": code, "evidence_text": evidence_cache[code], "fetched_at": datetime.now()})
|
||||
if len(evidence_rows) >= 50:
|
||||
batch_upsert(StockF10Evidence, evidence_rows, ["stock_code"])
|
||||
evidence_rows.clear()
|
||||
except Exception as exc:
|
||||
try:
|
||||
fallback_evidence = fetch_ths_stock_industry_evidence(code)
|
||||
if not fallback_evidence:
|
||||
raise ValueError("公司概况无行业字段")
|
||||
evidence_cache[code] = fallback_evidence
|
||||
evidence_rows.append({"stock_code": code, "evidence_text": fallback_evidence, "fetched_at": datetime.now()})
|
||||
_logger.info("活跃票 %s 主营页缺失，已使用同花顺申万行业补充", code)
|
||||
except Exception as fallback_exc:
|
||||
failed.add(code)
|
||||
_logger.warning("活跃票 %s F10获取失败: %s；行业补充失败: %s", code, exc, fallback_exc)
|
||||
if completed % 50 == 0 or completed == len(fetch_codes):
|
||||
_logger.info("活跃票 F10抓取 %d/%d，失败:%d", completed, len(fetch_codes), len(failed))
|
||||
if evidence_rows:
|
||||
batch_upsert(StockF10Evidence, evidence_rows, ["stock_code"])
|
||||
|
||||
membership_rows: list[dict[str, Any]] = []
|
||||
matched_codes: set[str] = set()
|
||||
for code in codes:
|
||||
evidence = evidence_cache.get(code, "")
|
||||
if code in failed or not evidence:
|
||||
continue
|
||||
for spec in CHAIN_SPECS:
|
||||
_, profile_fields = _split_f10_profile_evidence(evidence)
|
||||
allow_business_default = spec.key == "F10主营" and any(profile_fields)
|
||||
industry_codes = classify_chain_profiles(None, spec, evidence, allow_default=allow_business_default)
|
||||
if not industry_codes:
|
||||
continue
|
||||
matched_codes.add(code)
|
||||
for industry_code in industry_codes:
|
||||
membership_rows.append({
|
||||
"source": spec.source, "stock_code": code, "industry_code": industry_code,
|
||||
"effective_date": None, "updated_at": checked_at,
|
||||
})
|
||||
replace_stock_industry_memberships(membership_rows)
|
||||
_logger.info("活跃票批量分类完成，候选:%d，成功:%d，未匹配:%d，失败:%d", len(codes), len(matched_codes), len(codes) - len(matched_codes), len(failed))
|
||||
return len(matched_codes), len(codes) - len(matched_codes), len(failed)
|
||||
|
||||
|
||||
def sync_semiconductor_industry_chain(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
taxonomy_only: bool = False,
|
||||
refresh: bool = False,
|
||||
workers: int = 4,
|
||||
) -> tuple[int, int, int]:
|
||||
return sync_industry_chains(
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
refresh=refresh,
|
||||
taxonomy_only=taxonomy_only,
|
||||
workers=workers,
|
||||
)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"CHAIN_SPECS",
|
||||
"INDUSTRY_CHAIN_SOURCE",
|
||||
"classify_chain_profile",
|
||||
"classify_chain_profiles",
|
||||
"classify_semiconductor_profile",
|
||||
"fetch_ths_board_constituents",
|
||||
"fetch_ths_semiconductor_constituents",
|
||||
"fetch_ths_stock_concept_evidence",
|
||||
"fetch_ths_stock_concepts",
|
||||
"parse_ths_concept_evidence_html",
|
||||
"parse_ths_concepts_html",
|
||||
"semiconductor_chain_nodes",
|
||||
"sync_industry_chains",
|
||||
"classify_unclassified_active",
|
||||
"sync_semiconductor_industry_chain",
|
||||
]
|
||||
@@ -1,341 +0,0 @@
|
||||
"""分钟K线抓取模块 — 通达信本地客户端。
|
||||
|
||||
用法：
|
||||
python -m src.main --intraday --freq 1 # 1分钟K线
|
||||
python -m src.main --intraday --freq 5 # 5分钟K线
|
||||
python -m src.main --intraday --start-date 20260508 --end-date 20260509
|
||||
python -m src.main --intraday --symbol 000001
|
||||
"""
|
||||
|
||||
import time
|
||||
import random
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import pandas as pd
|
||||
from src.config import get_fetch_config
|
||||
from src.db import StockMin1, StockMin5, TradingDay, batch_upsert, get_session, get_stock_codes
|
||||
from src.fetchers.tdx_client import get_market_data as tdx_get_market_data
|
||||
from src.fetchers.tdx_client import format_tdx_code_for_log
|
||||
from src.fetchers.tdx_client import normalize_tdx_code
|
||||
from src.fetchers.tdx_client import refresh_minute_cache as tdx_refresh_minute_cache
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import select, func, distinct, text
|
||||
|
||||
_logger = get_logger("intraday")
|
||||
|
||||
|
||||
VALID_FREQ = ("1", "5")
|
||||
|
||||
TDX_AUTOCACHE_PERIODS = ("1m", "5m")
|
||||
|
||||
FREQ_MODEL = {
|
||||
"1": StockMin1,
|
||||
"5": StockMin5,
|
||||
}
|
||||
|
||||
FREQ_TDX_PERIOD = {
|
||||
"1": "1m",
|
||||
"5": "5m",
|
||||
}
|
||||
|
||||
|
||||
def _clean(val):
|
||||
if val is None:
|
||||
return None
|
||||
if isinstance(val, str) and val.strip() == "":
|
||||
return None
|
||||
return val
|
||||
|
||||
|
||||
def _parse_datetime(date_str: str, time_str: str) -> str | None:
|
||||
"""将分钟线返回的 date + time 解析为 datetime 字符串
|
||||
|
||||
time 格式: "20260508093500000" (17位) 或 "09:35:00" (8位)
|
||||
"""
|
||||
if len(time_str) == 17:
|
||||
return f"{time_str[:4]}-{time_str[4:6]}-{time_str[6:8]} " \
|
||||
f"{time_str[8:10]}:{time_str[10:12]}:{time_str[12:14]}"
|
||||
return f"{date_str} {time_str}"
|
||||
|
||||
|
||||
def _parse_tdx_datetime(value) -> datetime:
|
||||
"""将通达信返回的时间索引转成 `datetime`。"""
|
||||
if isinstance(value, datetime):
|
||||
return value
|
||||
if hasattr(value, "to_pydatetime"):
|
||||
return value.to_pydatetime()
|
||||
return pd.to_datetime(value).to_pydatetime()
|
||||
|
||||
|
||||
def _tdx_market_data_to_rows(code: str, market_data: dict) -> list[dict]:
|
||||
"""把通达信 `get_market_data` 的返回值整理成数据库行。"""
|
||||
if not market_data:
|
||||
return []
|
||||
|
||||
lower_map = {str(key).lower(): value for key, value in market_data.items()}
|
||||
field_map = {
|
||||
"open": "open",
|
||||
"high": "high",
|
||||
"low": "low",
|
||||
"close": "close",
|
||||
"volume": "volume",
|
||||
"amount": "amount",
|
||||
}
|
||||
|
||||
base_frame = None
|
||||
for field in ("close", "open", "high", "low", "volume", "amount"):
|
||||
frame = lower_map.get(field)
|
||||
if hasattr(frame, "index") and len(frame.index) > 0:
|
||||
base_frame = frame
|
||||
break
|
||||
|
||||
if base_frame is None:
|
||||
return []
|
||||
|
||||
rows: list[dict] = []
|
||||
for ts in base_frame.index:
|
||||
row = {
|
||||
"code": code,
|
||||
"datetime": _parse_tdx_datetime(ts),
|
||||
}
|
||||
valid_value = False
|
||||
for field, column in field_map.items():
|
||||
frame = lower_map.get(field)
|
||||
if frame is None or ts not in frame.index:
|
||||
continue
|
||||
cell = frame.loc[ts]
|
||||
if isinstance(cell, pd.Series):
|
||||
if code in cell.index:
|
||||
value = cell[code]
|
||||
else:
|
||||
value = cell.iloc[0]
|
||||
else:
|
||||
value = cell
|
||||
value = _clean(value)
|
||||
row[column] = value
|
||||
if value is not None:
|
||||
valid_value = True
|
||||
if valid_value:
|
||||
rows.append(row)
|
||||
|
||||
return rows
|
||||
|
||||
|
||||
def _tdx_fetch_rows_with_autocache(
|
||||
code: str,
|
||||
tdx_code: str,
|
||||
tdx_period: str,
|
||||
start_time: str,
|
||||
end_time: str,
|
||||
) -> tuple[list[dict], bool]:
|
||||
"""先读取通达信本地数据，缺缓存时自动刷新一次后重试。"""
|
||||
def _fetch_rows(period: str) -> list[dict]:
|
||||
market_data = tdx_get_market_data(
|
||||
[tdx_code],
|
||||
period=period,
|
||||
start_time=start_time,
|
||||
end_time=end_time,
|
||||
count=-1,
|
||||
dividend_type="none",
|
||||
fill_data=True,
|
||||
)
|
||||
return _tdx_market_data_to_rows(code, market_data)
|
||||
|
||||
rows = _fetch_rows(tdx_period)
|
||||
if rows:
|
||||
return rows, False
|
||||
|
||||
_logger.info(
|
||||
"通达信 %s(%s) 首次没有返回数据，自动刷新本地分钟缓存后重试",
|
||||
code, tdx_code,
|
||||
)
|
||||
tdx_refresh_minute_cache(
|
||||
[tdx_code],
|
||||
periods=TDX_AUTOCACHE_PERIODS,
|
||||
batch_size=1,
|
||||
pause_seconds=0,
|
||||
)
|
||||
rows = _fetch_rows(tdx_period)
|
||||
if rows:
|
||||
return rows, True
|
||||
|
||||
return [], True
|
||||
|
||||
|
||||
def _get_intraday_gaps(model, code: str, sd: str, ed: str) -> list[tuple[str, str]]:
|
||||
"""分析单只股票在 [sd, ed] 范围内的分钟K线缺口，返回缺失区间列表"""
|
||||
session = get_session()
|
||||
try:
|
||||
# 获取范围内的交易日
|
||||
trading_days = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
).scalars().all()
|
||||
if not trading_days:
|
||||
return [(sd, ed)]
|
||||
|
||||
# 获取该股票已有数据的日期（按天去重）
|
||||
existing = set(session.execute(
|
||||
text(f"SELECT DISTINCT DATE(datetime) FROM {model.__tablename__} "
|
||||
"WHERE code = :code AND datetime >= :sd AND datetime <= :ed"),
|
||||
{"code": code, "sd": sd, "ed": ed + " 23:59:59"},
|
||||
).scalars().all())
|
||||
|
||||
# 找出缺失的交易日
|
||||
missing = [d for d in trading_days if d not in existing]
|
||||
if not missing:
|
||||
return []
|
||||
|
||||
# 合并为连续区间
|
||||
gaps = []
|
||||
gap_start = missing[0]
|
||||
gap_end = missing[0]
|
||||
for d in missing[1:]:
|
||||
if (d - gap_end).days <= 3:
|
||||
gap_end = d
|
||||
else:
|
||||
gaps.append((str(gap_start), str(gap_end)))
|
||||
gap_start = d
|
||||
gap_end = d
|
||||
gaps.append((str(gap_start), str(gap_end)))
|
||||
return gaps
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def fetch_intraday(start_date: str | None = None, end_date: str | None = None,
|
||||
symbol: str | None = None, freq: str = "5",
|
||||
prewarm_cache: bool = True):
|
||||
"""抓取分钟K线行情
|
||||
|
||||
Args:
|
||||
start_date: 开始日期 YYYYMMDD，默认30天前
|
||||
end_date: 结束日期 YYYYMMDD，默认今天
|
||||
symbol: 单只股票代码，默认全部
|
||||
freq: K线频率 1 或 5
|
||||
prewarm_cache: 是否先批量刷新通达信本地分钟缓存
|
||||
"""
|
||||
if freq not in VALID_FREQ:
|
||||
_logger.error("不支持的频率 %s，可选: %s", freq, ", ".join(VALID_FREQ))
|
||||
return
|
||||
|
||||
_fetch_one_freq_tdx(freq, start_date, end_date, symbol, prewarm_cache=prewarm_cache)
|
||||
|
||||
|
||||
def _fetch_one_freq_tdx(freq: str, start_date: str | None, end_date: str | None,
|
||||
symbol: str | None, prewarm_cache: bool = True):
|
||||
"""使用通达信本地客户端抓取单个频率的分钟K线。"""
|
||||
cfg = get_fetch_config()
|
||||
base_delay = float(cfg.get("delay", 0.1))
|
||||
delay = max(base_delay, 0.2)
|
||||
model = FREQ_MODEL[freq]
|
||||
tdx_period = FREQ_TDX_PERIOD[freq]
|
||||
|
||||
if end_date is None:
|
||||
end_date = datetime.now().strftime("%Y%m%d")
|
||||
if start_date is None:
|
||||
start_date = (datetime.now() - timedelta(days=30)).strftime("%Y%m%d")
|
||||
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
|
||||
if symbol:
|
||||
codes = [symbol]
|
||||
else:
|
||||
codes = get_stock_codes()
|
||||
if not codes:
|
||||
_logger.error("无股票列表，请先运行 --stock-info")
|
||||
return
|
||||
|
||||
need_fetch = []
|
||||
if symbol:
|
||||
need_fetch = [(symbol, [(sd, ed)])]
|
||||
else:
|
||||
skip = 0
|
||||
for code in codes:
|
||||
gaps = _get_intraday_gaps(model, code, sd, ed)
|
||||
if gaps:
|
||||
need_fetch.append((code, gaps))
|
||||
else:
|
||||
skip += 1
|
||||
codes = [c for c, _ in need_fetch]
|
||||
|
||||
if not need_fetch:
|
||||
_logger.info("通达信 %s分钟K线 %s ~ %s 数据已完整，跳过", freq, sd, ed)
|
||||
return
|
||||
|
||||
skip_msg = f"（跳过 {skip} 只已完整）" if not symbol and skip else ""
|
||||
preview = ", ".join(format_tdx_code_for_log(code) for code in codes[:8])
|
||||
if len(codes) > 8:
|
||||
preview += f", ...(+{len(codes) - 8})"
|
||||
_logger.info(
|
||||
"正在使用通达信抓取%s分钟K线 %s ~ %s，需补缺 %d 只%s，目标: [%s]",
|
||||
freq, sd, ed, len(need_fetch), skip_msg, preview,
|
||||
)
|
||||
|
||||
# 全市场抓取先批量预热，单只股票则走按需自动缓存，避免重复刷新。
|
||||
if not symbol and prewarm_cache:
|
||||
try:
|
||||
refresh_results = tdx_refresh_minute_cache(codes, periods=TDX_AUTOCACHE_PERIODS)
|
||||
refresh_failed = [item for item in refresh_results if item.get("result", {}).get("ErrorId") not in (None, "0", 0)]
|
||||
if refresh_failed:
|
||||
_logger.warning(
|
||||
"通达信分钟缓存刷新后仍有 %d 个批次失败，分钟线可能不完整",
|
||||
len(refresh_failed),
|
||||
)
|
||||
except Exception as exc:
|
||||
_logger.warning("通达信分钟缓存刷新失败，继续尝试取数：%s", exc)
|
||||
|
||||
total = len(need_fetch)
|
||||
success = 0
|
||||
fail = 0
|
||||
t_start = time.time()
|
||||
|
||||
for i, (code, gaps) in enumerate(need_fetch):
|
||||
total_rows = 0
|
||||
tdx_code = normalize_tdx_code(code)
|
||||
try:
|
||||
for gap_sd, gap_ed in gaps:
|
||||
start_time = f"{gap_sd.replace('-', '')}000000"
|
||||
end_time = f"{gap_ed.replace('-', '')}235959"
|
||||
rows, _refreshed = _tdx_fetch_rows_with_autocache(
|
||||
code,
|
||||
tdx_code,
|
||||
tdx_period,
|
||||
start_time,
|
||||
end_time,
|
||||
)
|
||||
if rows:
|
||||
batch_upsert(model, rows, ["code", "datetime"])
|
||||
total_rows += len(rows)
|
||||
time.sleep(delay + random.uniform(0, delay * 0.2))
|
||||
if total_rows:
|
||||
success += 1
|
||||
else:
|
||||
fail += 1
|
||||
_logger.warning(
|
||||
"通达信 %s分钟K线 %s(%s) 没有返回数据，区间 %s ~ %s，period=%s；"
|
||||
"通常表示本地客户端没有下载该周期的分钟缓存，或该股在此区间无分钟数据",
|
||||
freq, code, tdx_code, sd, ed, tdx_period,
|
||||
)
|
||||
except Exception as exc:
|
||||
fail += 1
|
||||
_logger.warning(
|
||||
"通达信 %s分钟K线 %s(%s) 失败: %s，区间 %s ~ %s，period=%s",
|
||||
freq, code, tdx_code, exc, sd, ed, tdx_period,
|
||||
)
|
||||
|
||||
if (i + 1) % 100 == 0:
|
||||
elapsed = time.time() - t_start
|
||||
_logger.info(
|
||||
"[%d/%d] 进度... 成功:%d 失败:%d 已用时:%.0fs",
|
||||
i + 1, total, success, fail, elapsed,
|
||||
)
|
||||
|
||||
total_time = time.time() - t_start
|
||||
_logger.info(
|
||||
"通达信 %s分钟K线抓取完成，成功:%d 失败:%d 总耗时:%.1fs",
|
||||
freq, success, fail, total_time,
|
||||
)
|
||||
@@ -1,111 +0,0 @@
|
||||
"""每日涨跌停统计 — 从 stock_daily 汇总
|
||||
|
||||
按板块区分：
|
||||
- 10% 涨跌停：主板（沪60x/深00x）
|
||||
- 20% 涨跌停：科创板（688/689）、创业板（300/301）
|
||||
|
||||
用法：
|
||||
python -m src.main --market-daily
|
||||
python -m src.main --market-daily --start-date 20260101 --end-date 20260509
|
||||
"""
|
||||
|
||||
from datetime import datetime
|
||||
from sqlalchemy import select, func, case, and_, text
|
||||
from src.db import StockDaily, MarketDaily, batch_upsert, get_session
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("market_daily")
|
||||
|
||||
|
||||
# 20% 板块：300/301 创业板，688/689 科创板
|
||||
def _is_20pct(code: str) -> bool:
|
||||
return code[:3] in ("300", "301", "688", "689")
|
||||
|
||||
|
||||
def _fetch_history(start_date: str | None, end_date: str | None):
|
||||
if end_date is None:
|
||||
end_date = datetime.now().strftime("%Y%m%d")
|
||||
if start_date is None:
|
||||
start_date = "19901219"
|
||||
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
|
||||
_logger.info("正在从 stock_daily 汇总涨跌停统计 %s ~ %s...", sd, ed)
|
||||
|
||||
# 跳过已有的日期
|
||||
session = get_session()
|
||||
try:
|
||||
existing = session.execute(
|
||||
select(MarketDaily.date)
|
||||
.where(MarketDaily.date >= sd)
|
||||
.where(MarketDaily.date <= ed)
|
||||
)
|
||||
existing_dates = {str(row[0]) for row in existing}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
# 一次性查出所有需处理的代码前缀信息，按日聚合
|
||||
session = get_session()
|
||||
try:
|
||||
# 用 SQL 直接按日期分组，区分10%/20%板块
|
||||
sql = text("""
|
||||
SELECT
|
||||
date,
|
||||
SUM(CASE WHEN pct_change >= 9.8
|
||||
AND code NOT LIKE '300%%'
|
||||
AND code NOT LIKE '301%%'
|
||||
AND code NOT LIKE '688%%'
|
||||
AND code NOT LIKE '689%%'
|
||||
THEN 1 ELSE 0 END) AS limit_up_10,
|
||||
SUM(CASE WHEN pct_change >= 19.5
|
||||
AND (code LIKE '300%%' OR code LIKE '301%%'
|
||||
OR code LIKE '688%%' OR code LIKE '689%%')
|
||||
THEN 1 ELSE 0 END) AS limit_up_20,
|
||||
SUM(CASE WHEN pct_change <= -9.8
|
||||
AND code NOT LIKE '300%%'
|
||||
AND code NOT LIKE '301%%'
|
||||
AND code NOT LIKE '688%%'
|
||||
AND code NOT LIKE '689%%'
|
||||
THEN 1 ELSE 0 END) AS limit_down_10,
|
||||
SUM(CASE WHEN pct_change <= -19.5
|
||||
AND (code LIKE '300%%' OR code LIKE '301%%'
|
||||
OR code LIKE '688%%' OR code LIKE '689%%')
|
||||
THEN 1 ELSE 0 END) AS limit_down_20
|
||||
FROM stock_daily
|
||||
WHERE date >= :sd AND date <= :ed
|
||||
AND pct_change IS NOT NULL
|
||||
GROUP BY date
|
||||
ORDER BY date
|
||||
""")
|
||||
result = session.execute(sql, {"sd": sd, "ed": ed})
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
rows = []
|
||||
for row in result:
|
||||
date_str = str(row.date)
|
||||
if date_str in existing_dates:
|
||||
continue
|
||||
rows.append({
|
||||
"date": date_str,
|
||||
"limit_up_10": int(row.limit_up_10 or 0),
|
||||
"limit_up_20": int(row.limit_up_20 or 0),
|
||||
"limit_down_10": int(row.limit_down_10 or 0),
|
||||
"limit_down_20": int(row.limit_down_20 or 0),
|
||||
})
|
||||
|
||||
if rows:
|
||||
batch_upsert(MarketDaily, rows, ["date"])
|
||||
_logger.info("已写入 %d 天涨跌停统计（%s ~ %s）", len(rows), rows[0]['date'], rows[-1]['date'])
|
||||
return rows
|
||||
|
||||
|
||||
def fetch_market_daily(start_date: str | None = None, end_date: str | None = None):
|
||||
"""从 stock_daily 汇总每日涨跌停统计并写入 market_daily 表
|
||||
|
||||
Args:
|
||||
start_date: 开始日期 YYYYMMDD，默认 19901219
|
||||
end_date: 结束日期 YYYYMMDD，默认今天
|
||||
"""
|
||||
return _fetch_history(start_date, end_date)
|
||||
@@ -1,86 +0,0 @@
|
||||
"""概念板块数据抓取 — 通达信本地缓存。"""
|
||||
|
||||
import time
|
||||
|
||||
from src.db import StockConcept, batch_upsert, get_session
|
||||
from src.fetchers.tdx_blocks import load_infoharbor_blocks
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import func, select
|
||||
|
||||
_logger = get_logger("sector")
|
||||
|
||||
|
||||
def _is_a_stock_code(code: str) -> bool:
|
||||
return len(code) == 6 and code.isdigit() and not code.startswith("920")
|
||||
|
||||
|
||||
def _fetch_concept_list() -> list[dict]:
|
||||
"""获取全部概念板块列表。"""
|
||||
return _fetch_concept_list_tdx()
|
||||
|
||||
|
||||
def _fetch_concept_list_tdx() -> list[dict]:
|
||||
"""使用通达信本地概念板块缓存获取全部板块。"""
|
||||
blocks = load_infoharbor_blocks()
|
||||
return [{"code": block.code, "name": block.name} for block in blocks if block.stocks]
|
||||
|
||||
|
||||
def _fetch_concept_stocks(concept_code: str) -> list[str]:
|
||||
"""获取单个概念板块的成分股代码列表（仅保留 A 股）。"""
|
||||
return _fetch_concept_stocks_tdx(concept_code)
|
||||
|
||||
|
||||
def _fetch_concept_stocks_tdx(concept_code: str) -> list[str]:
|
||||
"""使用通达信本地概念板块缓存获取成分股。"""
|
||||
for block in load_infoharbor_blocks():
|
||||
if concept_code == block.code or concept_code == block.name:
|
||||
return [code for code in block.stocks if _is_a_stock_code(code)]
|
||||
return []
|
||||
|
||||
|
||||
def fetch_sector():
|
||||
"""抓取全部概念板块及成分股，写入 stock_concept 表。"""
|
||||
_logger.info("正在获取概念板块列表...")
|
||||
concepts = _fetch_concept_list()
|
||||
if not concepts:
|
||||
_logger.warning("概念板块列表获取失败")
|
||||
return
|
||||
|
||||
session = get_session()
|
||||
try:
|
||||
existing_count = session.execute(
|
||||
select(func.count(StockConcept.id))
|
||||
).scalar() or 0
|
||||
existing_concepts = session.execute(
|
||||
select(func.count(func.distinct(StockConcept.concept_code)))
|
||||
).scalar() or 0
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
if existing_concepts >= len(concepts):
|
||||
_logger.info("概念板块已有 %d 个概念、%d 条记录，跳过", existing_concepts, existing_count)
|
||||
return
|
||||
|
||||
_logger.info("共 %d 个概念板块，开始抓取成分股...", len(concepts))
|
||||
t_start = time.time()
|
||||
rows = []
|
||||
for i, concept in enumerate(concepts):
|
||||
stocks = _fetch_concept_stocks(concept["code"])
|
||||
for code in stocks:
|
||||
rows.append({
|
||||
"code": code,
|
||||
"concept_code": concept["code"],
|
||||
"concept_name": concept["name"],
|
||||
})
|
||||
if (i + 1) % 50 == 0:
|
||||
elapsed = time.time() - t_start
|
||||
_logger.info("[%d/%d] 已处理 耗时:%.0fs", i + 1, len(concepts), elapsed)
|
||||
|
||||
if rows:
|
||||
batch_upsert(StockConcept, rows, ["code", "concept_code"])
|
||||
_logger.info(
|
||||
"概念板块写入完成，%d 个概念，%d 条记录，耗时:%.0fs",
|
||||
len(concepts), len(rows), time.time() - t_start,
|
||||
)
|
||||
else:
|
||||
_logger.warning("无概念板块数据")
|
||||
+5
-113
@@ -1,120 +1,12 @@
|
||||
"""股票列表抓取模块 — AKShare。"""
|
||||
"""兼容入口：股票列表统一使用当前行情实现。"""
|
||||
|
||||
import random
|
||||
import time
|
||||
from datetime import datetime
|
||||
from __future__ import annotations
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import (
|
||||
StockInfo,
|
||||
batch_upsert,
|
||||
get_session,
|
||||
invalidate_ipo_dates_cache,
|
||||
invalidate_stock_codes_cache,
|
||||
)
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import func, select
|
||||
|
||||
_logger = get_logger("stock_list")
|
||||
|
||||
try:
|
||||
import akshare as ak
|
||||
except Exception: # pragma: no cover - 运行环境缺少依赖时再报错
|
||||
ak = None
|
||||
|
||||
|
||||
def _parse_ipo_date(raw: str | None):
|
||||
if not raw:
|
||||
return None
|
||||
raw = str(raw).strip()
|
||||
if not raw:
|
||||
return None
|
||||
if len(raw) == 8 and raw.isdigit():
|
||||
return datetime.strptime(raw, "%Y%m%d").date()
|
||||
if len(raw) >= 10 and raw[4] == "-" and raw[7] == "-":
|
||||
try:
|
||||
return datetime.strptime(raw[:10], "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _fetch_akshare_stock_list(*, retry: int = 3, delay: float = 0.1) -> list[dict]:
|
||||
"""从 AKShare 获取沪深 A 股代码与名称。"""
|
||||
if ak is None:
|
||||
_logger.error("未安装 akshare，无法抓取股票列表")
|
||||
return []
|
||||
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
df = ak.stock_info_a_code_name()
|
||||
if df is None or df.empty:
|
||||
return []
|
||||
if "code" not in df.columns or "name" not in df.columns:
|
||||
_logger.warning("AKShare 股票列表字段不完整：%s", list(df.columns))
|
||||
return []
|
||||
rows: list[dict] = []
|
||||
for _, item in df.iterrows():
|
||||
code = str(item.get("code") or "").strip()
|
||||
name = str(item.get("name") or "").strip()
|
||||
if not code or not name:
|
||||
continue
|
||||
rows.append({"code": code, "name": name})
|
||||
return rows
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
break
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
wait_seconds += random.uniform(0, min(wait_seconds * 0.2, 0.5))
|
||||
_logger.warning("AKShare 获取股票列表第 %d 次失败，%.1fs 后重试: %s", attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
if last_exc is not None:
|
||||
_logger.warning("AKShare 获取股票列表失败: %s", last_exc)
|
||||
return []
|
||||
from src.fetchers.tdx_daily import bootstrap_stock_list, fetch_stock_list_rows
|
||||
|
||||
|
||||
def fetch_stock_list():
|
||||
"""从 AKShare 读取沪深 A 股列表，upsert 到 stock_info 表。"""
|
||||
session = get_session()
|
||||
try:
|
||||
existing = session.execute(select(func.count(StockInfo.code))).scalar() or 0
|
||||
finally:
|
||||
session.close()
|
||||
return bootstrap_stock_list()
|
||||
|
||||
cfg = get_fetch_config()
|
||||
_logger.info("正在抓取股票列表（已有 %d 条）...", existing)
|
||||
rows = _fetch_akshare_stock_list(retry=cfg.get("retry", 3), delay=cfg.get("delay", 0.1))
|
||||
if not rows:
|
||||
_logger.warning("AKShare 股票列表为空")
|
||||
return
|
||||
|
||||
upsert_rows: list[dict] = []
|
||||
for index, row in enumerate(rows, start=1):
|
||||
code = str(row.get("code") or "").strip()
|
||||
name = str(row.get("name") or "").strip()
|
||||
if not code or not name:
|
||||
continue
|
||||
item = {
|
||||
"code": code,
|
||||
"name": name,
|
||||
}
|
||||
ipo_date = _parse_ipo_date(row.get("ipo_date"))
|
||||
if ipo_date is not None:
|
||||
item["ipo_date"] = ipo_date
|
||||
upsert_rows.append(item)
|
||||
if index % 500 == 0:
|
||||
_logger.info("已解析 %d/%d 只股票", index, len(rows))
|
||||
|
||||
if upsert_rows:
|
||||
batch_upsert(StockInfo, upsert_rows, ["code"])
|
||||
invalidate_stock_codes_cache()
|
||||
invalidate_ipo_dates_cache()
|
||||
_logger.info("股票列表已更新，共 %d 只", len(upsert_rows))
|
||||
else:
|
||||
_logger.warning("无股票数据")
|
||||
__all__ = ["fetch_stock_list", "fetch_stock_list_rows"]
|
||||
|
||||
@@ -0,0 +1,749 @@
|
||||
"""日线行情抓取实现。
|
||||
|
||||
通过多主机轮询降低单个行情服务器不可用时的失败率。
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime, timedelta
|
||||
from typing import Iterator
|
||||
|
||||
import pandas as pd
|
||||
from sqlalchemy import select
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import (
|
||||
StockDaily,
|
||||
StockInfo,
|
||||
TradingDay,
|
||||
batch_upsert,
|
||||
get_ipo_dates,
|
||||
get_session,
|
||||
get_stock_codes,
|
||||
invalidate_ipo_dates_cache,
|
||||
invalidate_stock_codes_cache,
|
||||
)
|
||||
from src.db.status import get_suspended_trade_days
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("daily")
|
||||
|
||||
TDX_DAILY_CATEGORY = 9
|
||||
TDX_PAGE_SIZE = 800
|
||||
TDX_SECURITY_PAGE_SIZE = 1000
|
||||
TDX_MAX_PAGES = 20
|
||||
TDX_DEFAULT_TIMEOUT = 8
|
||||
TDX_DEFAULT_HOSTS = (
|
||||
("119.147.212.81", 7709),
|
||||
("106.14.95.149", 7709),
|
||||
("180.153.18.170", 7709),
|
||||
("115.238.56.198", 7709),
|
||||
("202.108.253.130", 7709),
|
||||
("14.17.75.71", 7709),
|
||||
)
|
||||
|
||||
|
||||
class TdxUnavailableError(RuntimeError):
|
||||
"""所有行情服务器均不可用。"""
|
||||
|
||||
|
||||
def _normalize_code(code: str) -> str:
|
||||
text = (code or "").strip()
|
||||
if "." in text:
|
||||
base, suffix = text.upper().split(".", 1)
|
||||
if suffix in {"SH", "SZ"}:
|
||||
return base
|
||||
lower = text.lower()
|
||||
if lower.startswith(("sh.", "sz.")):
|
||||
return lower[3:]
|
||||
if lower.startswith(("sh", "sz")):
|
||||
return lower[2:].lstrip(".")
|
||||
return text
|
||||
|
||||
|
||||
def _parse_symbol_list(symbol: str | None) -> list[str]:
|
||||
if not symbol:
|
||||
return []
|
||||
return [
|
||||
_normalize_code(item)
|
||||
for item in symbol.replace("，", ",").split(",")
|
||||
if _normalize_code(item)
|
||||
]
|
||||
|
||||
|
||||
def _preview_codes(codes: list[str], limit: int = 10) -> str:
|
||||
if not codes:
|
||||
return "[]"
|
||||
preview = codes[:limit]
|
||||
suffix = ""
|
||||
if len(codes) > limit:
|
||||
suffix = f", ...(+{len(codes) - limit})"
|
||||
return "[" + ", ".join(preview) + suffix + "]"
|
||||
|
||||
|
||||
def infer_tdx_market(code: str) -> int:
|
||||
"""行情市场：0=深市，1=沪市。"""
|
||||
db_code = _normalize_code(code)
|
||||
if len(db_code) != 6 or not db_code.isdigit():
|
||||
raise ValueError(f"仅支持 6 位沪深代码: {code}")
|
||||
if db_code.startswith("6"):
|
||||
return 1
|
||||
if db_code.startswith("0") or db_code.startswith("30"):
|
||||
return 0
|
||||
raise ValueError(f"日线跳过非沪深主板/创业板代码: {code}")
|
||||
|
||||
|
||||
def _parse_host_text(value: str) -> tuple[str, int] | None:
|
||||
text = str(value or "").strip()
|
||||
if not text:
|
||||
return None
|
||||
if ":" not in text:
|
||||
return text, 7709
|
||||
host, port_text = text.rsplit(":", 1)
|
||||
try:
|
||||
return host.strip(), int(port_text)
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _configured_hosts() -> list[tuple[str, int]]:
|
||||
try:
|
||||
cfg = get_fetch_config()
|
||||
except Exception:
|
||||
cfg = {}
|
||||
raw_hosts = cfg.get("market_hosts", cfg.get("tdx_hosts"))
|
||||
if isinstance(raw_hosts, str):
|
||||
hosts = [_parse_host_text(item) for item in raw_hosts.split(",")]
|
||||
return [item for item in hosts if item]
|
||||
if isinstance(raw_hosts, list):
|
||||
hosts = [_parse_host_text(item) for item in raw_hosts]
|
||||
return [item for item in hosts if item]
|
||||
return []
|
||||
|
||||
|
||||
def _pytdx_hosts() -> list[tuple[str, int]]:
|
||||
try:
|
||||
from pytdx.config.hosts import hq_hosts
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
hosts: list[tuple[str, int]] = []
|
||||
for item in hq_hosts:
|
||||
try:
|
||||
if len(item) >= 3:
|
||||
hosts.append((str(item[1]), int(item[2])))
|
||||
except Exception:
|
||||
continue
|
||||
return hosts
|
||||
|
||||
|
||||
def _iter_hosts() -> list[tuple[str, int]]:
|
||||
seen: set[tuple[str, int]] = set()
|
||||
hosts: list[tuple[str, int]] = []
|
||||
for host in _configured_hosts() + _pytdx_hosts() + list(TDX_DEFAULT_HOSTS):
|
||||
if host in seen:
|
||||
continue
|
||||
seen.add(host)
|
||||
hosts.append(host)
|
||||
return hosts
|
||||
|
||||
|
||||
def _tdx_timeout() -> int:
|
||||
try:
|
||||
cfg = get_fetch_config()
|
||||
return max(int(cfg.get("market_timeout", cfg.get("tdx_timeout", TDX_DEFAULT_TIMEOUT))), 1)
|
||||
except Exception:
|
||||
return TDX_DEFAULT_TIMEOUT
|
||||
|
||||
|
||||
@contextmanager
|
||||
def connected_session():
|
||||
"""连接可用的行情服务器。"""
|
||||
try:
|
||||
from pytdx.hq import TdxHq_API
|
||||
except Exception as exc:
|
||||
raise TdxUnavailableError("未安装 pytdx，请先安装依赖：pip install pytdx") from exc
|
||||
|
||||
last_exc: Exception | None = None
|
||||
for host, port in _iter_hosts():
|
||||
api = TdxHq_API(heartbeat=True, auto_retry=True, raise_exception=True)
|
||||
try:
|
||||
connected = api.connect(host, port, time_out=_tdx_timeout())
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
try:
|
||||
api.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
_logger.debug("行情服务器 %s:%s 连接失败: %s", host, port, exc)
|
||||
continue
|
||||
|
||||
if not connected:
|
||||
last_exc = TdxUnavailableError(f"{host}:{port} connect returned false")
|
||||
try:
|
||||
api.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
continue
|
||||
|
||||
try:
|
||||
yield api
|
||||
return
|
||||
finally:
|
||||
try:
|
||||
api.disconnect()
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
raise TdxUnavailableError(f"行情服务器均不可用: {last_exc}")
|
||||
|
||||
|
||||
def _bar_date(bar: dict) -> str | None:
|
||||
value = bar.get("datetime") or bar.get("date")
|
||||
if value is not None:
|
||||
parsed = pd.to_datetime(value, errors="coerce")
|
||||
if not pd.isna(parsed):
|
||||
return parsed.strftime("%Y-%m-%d")
|
||||
|
||||
year = bar.get("year")
|
||||
month = bar.get("month")
|
||||
day = bar.get("day")
|
||||
if year and month and day:
|
||||
try:
|
||||
return f"{int(year):04d}-{int(month):02d}-{int(day):02d}"
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _to_float(value):
|
||||
if value is None:
|
||||
return None
|
||||
try:
|
||||
parsed = float(value)
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
if pd.isna(parsed):
|
||||
return None
|
||||
return parsed
|
||||
|
||||
|
||||
def _tdx_bars_to_rows(
|
||||
code: str,
|
||||
bars: list[dict],
|
||||
*,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
) -> list[dict]:
|
||||
raw_rows: list[dict] = []
|
||||
for bar in bars:
|
||||
day = _bar_date(bar)
|
||||
if not day:
|
||||
continue
|
||||
row = {
|
||||
"code": _normalize_code(code),
|
||||
"date": day,
|
||||
"open": _to_float(bar.get("open")),
|
||||
"high": _to_float(bar.get("high")),
|
||||
"low": _to_float(bar.get("low")),
|
||||
"close": _to_float(bar.get("close")),
|
||||
"volume": _to_float(bar.get("vol", bar.get("volume"))),
|
||||
"turnover": _to_float(bar.get("amount")),
|
||||
"amplitude": None,
|
||||
"pct_change": None,
|
||||
"change": None,
|
||||
# pytdx K 线没有换手率；保持非空，避免 stock_daily 非空字段写入失败。
|
||||
"turnover_rate": 0.0,
|
||||
}
|
||||
if all(row.get(field) is not None for field in ("open", "high", "low", "close", "volume", "turnover")):
|
||||
raw_rows.append(row)
|
||||
|
||||
raw_rows.sort(key=lambda item: item["date"])
|
||||
dedup_rows = list({row["date"]: row for row in raw_rows}.values())
|
||||
for index, row in enumerate(dedup_rows):
|
||||
preclose = dedup_rows[index - 1]["close"] if index > 0 else None
|
||||
close = row["close"]
|
||||
high = row["high"]
|
||||
low = row["low"]
|
||||
if preclose is None or float(preclose) == 0:
|
||||
continue
|
||||
row["pct_change"] = round((float(close) - float(preclose)) / float(preclose) * 100, 2)
|
||||
row["change"] = round(float(close) - float(preclose), 3)
|
||||
row["amplitude"] = round((float(high) - float(low)) / float(preclose) * 100, 2)
|
||||
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
end_text = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
return [
|
||||
row for row in dedup_rows
|
||||
if start_text <= row["date"] <= end_text
|
||||
and all(row.get(field) is not None for field in (
|
||||
"open",
|
||||
"close",
|
||||
"high",
|
||||
"low",
|
||||
"volume",
|
||||
"turnover",
|
||||
"amplitude",
|
||||
"pct_change",
|
||||
"change",
|
||||
"turnover_rate",
|
||||
))
|
||||
]
|
||||
|
||||
|
||||
def _fetch_bars_payload(api, market: int, code: str, start_date: str) -> list[dict]:
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
rows: list[dict] = []
|
||||
for page in range(TDX_MAX_PAGES):
|
||||
offset = page * TDX_PAGE_SIZE
|
||||
payload = api.get_security_bars(TDX_DAILY_CATEGORY, market, code, offset, TDX_PAGE_SIZE)
|
||||
if not payload:
|
||||
break
|
||||
rows.extend(payload)
|
||||
dates = [day for day in (_bar_date(item) for item in payload) if day]
|
||||
if dates and min(dates) <= start_text:
|
||||
break
|
||||
return rows
|
||||
|
||||
|
||||
def _fetch_index_bars_payload(api, market: int, code: str, start_date: str) -> list[dict]:
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
rows: list[dict] = []
|
||||
for page in range(TDX_MAX_PAGES):
|
||||
offset = page * TDX_PAGE_SIZE
|
||||
payload = api.get_index_bars(TDX_DAILY_CATEGORY, market, code, offset, TDX_PAGE_SIZE)
|
||||
if not payload:
|
||||
break
|
||||
rows.extend(payload)
|
||||
dates = [day for day in (_bar_date(item) for item in payload) if day]
|
||||
if dates and min(dates) <= start_text:
|
||||
break
|
||||
return rows
|
||||
|
||||
|
||||
def fetch_one_stock(code: str, start_date: str, end_date: str) -> list[dict]:
|
||||
"""获取单只股票日线。"""
|
||||
db_code = _normalize_code(code)
|
||||
market = infer_tdx_market(db_code)
|
||||
with connected_session() as api:
|
||||
bars = _fetch_bars_payload(api, market, db_code, start_date)
|
||||
return _tdx_bars_to_rows(db_code, bars, start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
def _fetch_one_stock_with_api(api, code: str, start_date: str, end_date: str) -> list[dict]:
|
||||
db_code = _normalize_code(code)
|
||||
market = infer_tdx_market(db_code)
|
||||
bars = _fetch_bars_payload(api, market, db_code, start_date)
|
||||
return _tdx_bars_to_rows(db_code, bars, start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
def _fetch_one_stock_with_api_retry(
|
||||
api,
|
||||
code: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
retry: int = 3,
|
||||
delay: float = 0.1,
|
||||
) -> list[dict]:
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
return _fetch_one_stock_with_api(api, code, start_date, end_date)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
raise
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
_logger.warning("日线 %s 第 %d 次失败，%.1fs 后重试: %s", code, attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
if last_exc:
|
||||
raise last_exc
|
||||
return []
|
||||
|
||||
|
||||
def _is_tdx_session_error(exc: Exception) -> bool:
|
||||
text = str(exc).lower()
|
||||
return any(
|
||||
marker in text
|
||||
for marker in (
|
||||
"calling function error",
|
||||
"connection",
|
||||
"timeout",
|
||||
"socket",
|
||||
"10054",
|
||||
"10053",
|
||||
"broken pipe",
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def fetch_one_stock_with_retry(
|
||||
code: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
retry: int = 3,
|
||||
delay: float = 0.1,
|
||||
) -> list[dict]:
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
return fetch_one_stock(code, start_date, end_date)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
raise
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
_logger.warning("日线 %s 第 %d 次失败，%.1fs 后重试: %s", code, attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
if last_exc:
|
||||
raise last_exc
|
||||
return []
|
||||
|
||||
|
||||
def _is_a_share_code(market: int, code: str) -> bool:
|
||||
return len(code) == 6 and code.isdigit() and (
|
||||
(market == 1 and code.startswith("6"))
|
||||
or (market == 0 and (code.startswith("0") or code.startswith("30")))
|
||||
)
|
||||
|
||||
|
||||
def fetch_stock_list_rows() -> list[dict]:
|
||||
"""通过证券列表补充 stock_info。"""
|
||||
rows: list[dict] = []
|
||||
failed_pages: list[str] = []
|
||||
with connected_session() as api:
|
||||
for market in (0, 1):
|
||||
try:
|
||||
total = int(api.get_security_count(market) or 0)
|
||||
except Exception as exc:
|
||||
_logger.warning("股票列表市场 %s 数量获取失败，跳过该市场: %s", market, exc)
|
||||
continue
|
||||
offset = 0
|
||||
while offset < total:
|
||||
try:
|
||||
payload = api.get_security_list(market, offset)
|
||||
except UnicodeDecodeError as exc:
|
||||
failed_pages.append(f"{market}:{offset}")
|
||||
_logger.warning("股票列表市场 %s offset %s 解码失败，跳过该页: %s", market, offset, exc)
|
||||
offset += TDX_SECURITY_PAGE_SIZE
|
||||
continue
|
||||
except Exception as exc:
|
||||
failed_pages.append(f"{market}:{offset}")
|
||||
_logger.warning("股票列表市场 %s offset %s 获取失败，跳过该页: %s", market, offset, exc)
|
||||
offset += TDX_SECURITY_PAGE_SIZE
|
||||
continue
|
||||
if payload:
|
||||
for item in payload:
|
||||
code = str(item.get("code") or "").strip()
|
||||
if not _is_a_share_code(market, code):
|
||||
continue
|
||||
rows.append({
|
||||
"code": code,
|
||||
"name": str(item.get("name") or "").strip(),
|
||||
"ipo_date": None,
|
||||
})
|
||||
offset += TDX_SECURITY_PAGE_SIZE
|
||||
if failed_pages:
|
||||
_logger.warning("股票列表跳过异常分页 %d 页: %s", len(failed_pages), ", ".join(failed_pages[:10]))
|
||||
return list({row["code"]: row for row in rows}.values())
|
||||
|
||||
|
||||
def bootstrap_stock_list() -> list[str]:
|
||||
rows = fetch_stock_list_rows()
|
||||
if not rows:
|
||||
return []
|
||||
batch_upsert(StockInfo, rows, ["code"])
|
||||
invalidate_stock_codes_cache()
|
||||
invalidate_ipo_dates_cache()
|
||||
_logger.info("股票列表写入完成，%d 只", len(rows))
|
||||
return [row["code"] for row in rows]
|
||||
|
||||
|
||||
def fetch_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
"""用上证指数日线推导交易日，并写入 trading_day。"""
|
||||
with connected_session() as api:
|
||||
bars = _fetch_index_bars_payload(api, 1, "000001", start_date)
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
end_text = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
days = sorted({
|
||||
day for day in (_bar_date(item) for item in bars)
|
||||
if day and start_text <= day <= end_text
|
||||
})
|
||||
if days:
|
||||
batch_upsert(TradingDay, [{"date": day} for day in days], ["date"])
|
||||
_logger.info("交易日历写入完成，%s ~ %s 共 %d 天", days[0], days[-1], len(days))
|
||||
return days
|
||||
|
||||
|
||||
def _default_start_date() -> str:
|
||||
return (datetime.now() - timedelta(days=30)).strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _default_end_date() -> str:
|
||||
return datetime.now().strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _yyyymmdd_to_iso(value: str) -> str:
|
||||
return f"{value[:4]}-{value[4:6]}-{value[6:8]}"
|
||||
|
||||
|
||||
def _load_existing_daily_dates(codes: list[str], start_day: str, end_day: str) -> dict[str, set[str]]:
|
||||
if not codes:
|
||||
return {}
|
||||
|
||||
existing: dict[str, set[str]] = {}
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(
|
||||
select(StockDaily.code, StockDaily.date)
|
||||
.where(StockDaily.code.in_(codes))
|
||||
.where(StockDaily.date >= start_day)
|
||||
.where(StockDaily.date <= end_day)
|
||||
)
|
||||
for code, day in result:
|
||||
existing.setdefault(str(code), set()).add(str(day))
|
||||
finally:
|
||||
session.close()
|
||||
return existing
|
||||
|
||||
|
||||
def _split_contiguous_trading_day_ranges(
|
||||
days: list[str],
|
||||
trading_days: list[str],
|
||||
) -> list[tuple[str, str]]:
|
||||
if not days:
|
||||
return []
|
||||
|
||||
trading_index = {day: idx for idx, day in enumerate(trading_days)}
|
||||
sorted_days = sorted(dict.fromkeys(days), key=lambda day: trading_index.get(day, 10**9))
|
||||
ranges: list[tuple[str, str]] = []
|
||||
start_day = sorted_days[0]
|
||||
previous_day = sorted_days[0]
|
||||
previous_index = trading_index.get(previous_day)
|
||||
|
||||
for day in sorted_days[1:]:
|
||||
day_index = trading_index.get(day)
|
||||
if previous_index is not None and day_index == previous_index + 1:
|
||||
previous_day = day
|
||||
previous_index = day_index
|
||||
continue
|
||||
ranges.append((start_day, previous_day))
|
||||
start_day = previous_day = day
|
||||
previous_index = day_index
|
||||
|
||||
ranges.append((start_day, previous_day))
|
||||
return ranges
|
||||
|
||||
|
||||
def _supported_tdx_codes(codes: list[str]) -> tuple[list[str], list[str]]:
|
||||
supported: list[str] = []
|
||||
skipped: list[str] = []
|
||||
for code in codes:
|
||||
db_code = _normalize_code(code)
|
||||
try:
|
||||
infer_tdx_market(db_code)
|
||||
supported.append(db_code)
|
||||
except ValueError:
|
||||
skipped.append(db_code)
|
||||
return list(dict.fromkeys(supported)), list(dict.fromkeys(skipped))
|
||||
|
||||
|
||||
def fetch_tdx_daily(
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
"""批量抓取日线行情。"""
|
||||
cfg = get_fetch_config()
|
||||
retry = max(int(cfg.get("retry", 3)), 1)
|
||||
delay = max(float(cfg.get("tdx_daily_delay", cfg.get("delay", 0.1))), 0.0)
|
||||
progress_every = max(int(cfg.get("daily_progress_every", cfg.get("progress_every", 100))), 1)
|
||||
|
||||
start_date = start_date or _default_start_date()
|
||||
end_date = end_date or _default_end_date()
|
||||
start_day = _yyyymmdd_to_iso(start_date)
|
||||
end_day = _yyyymmdd_to_iso(end_date)
|
||||
|
||||
requested_codes = _parse_symbol_list(symbol)
|
||||
codes = requested_codes or get_stock_codes()
|
||||
if not codes:
|
||||
_logger.info("股票列表为空，先补齐股票列表后再下载日线")
|
||||
codes = bootstrap_stock_list()
|
||||
codes, skipped_codes = _supported_tdx_codes(codes)
|
||||
if skipped_codes:
|
||||
_logger.info("日线跳过不支持代码 %d 只: %s", len(skipped_codes), _preview_codes(skipped_codes))
|
||||
if not codes:
|
||||
_logger.warning("日线没有可抓取股票")
|
||||
return
|
||||
|
||||
trading_days = fetch_trading_days(start_date, end_date)
|
||||
if not trading_days:
|
||||
_logger.warning("区间内没有交易日，日线无需抓取")
|
||||
return
|
||||
|
||||
ipo_dates = get_ipo_dates()
|
||||
existing_by_code = _load_existing_daily_dates(codes, start_day, end_day)
|
||||
suspended_by_code = get_suspended_trade_days(codes, start_date, end_date) if skip_suspended else {}
|
||||
pending: list[tuple[str, list[tuple[str, str]]]] = []
|
||||
suspended_day_count = 0
|
||||
for code in codes:
|
||||
ipo = ipo_dates.get(code)
|
||||
expected_days = [
|
||||
day for day in trading_days
|
||||
if (not ipo or day >= ipo) and day not in existing_by_code.get(code, set())
|
||||
]
|
||||
suspended_days = suspended_by_code.get(code, set())
|
||||
suspended_day_count += len(set(expected_days) & suspended_days)
|
||||
expected_days = [day for day in expected_days if day not in suspended_days]
|
||||
if not expected_days:
|
||||
continue
|
||||
pending.append((code, _split_contiguous_trading_day_ranges(expected_days, trading_days)))
|
||||
if limit is not None and len(pending) >= max(int(limit), 1):
|
||||
break
|
||||
|
||||
_logger.info(
|
||||
"日线开始，区间:%s ~ %s，股票:%d，待抓取:%d，已完整:%d，停牌标记跳过:%d日，limit:%s",
|
||||
start_date,
|
||||
end_date,
|
||||
len(codes),
|
||||
len(pending),
|
||||
len(codes) - len(pending),
|
||||
suspended_day_count,
|
||||
limit if limit is not None else "all",
|
||||
)
|
||||
if not pending:
|
||||
return
|
||||
|
||||
success = 0
|
||||
fail = 0
|
||||
total_rows = 0
|
||||
write_batch_size = max(int(cfg.get("daily_write_batch_size", cfg.get("db_batch_size", 500))), 1)
|
||||
write_buffer: list[dict] = []
|
||||
failed_codes: list[str] = []
|
||||
t_start = time.time()
|
||||
total = len(pending)
|
||||
|
||||
def flush_rows(reason: str) -> None:
|
||||
if not write_buffer:
|
||||
return
|
||||
row_count = len(write_buffer)
|
||||
batch_upsert(StockDaily, write_buffer, ["code", "date"])
|
||||
write_buffer.clear()
|
||||
_logger.info("日线批量写入 %d 行，原因:%s，累计获取:%d 行", row_count, reason, total_rows)
|
||||
|
||||
api_cm = None
|
||||
api = None
|
||||
|
||||
def open_api():
|
||||
nonlocal api_cm, api
|
||||
if api_cm is not None:
|
||||
try:
|
||||
api_cm.__exit__(None, None, None)
|
||||
except Exception:
|
||||
pass
|
||||
api_cm = connected_session()
|
||||
api = api_cm.__enter__()
|
||||
|
||||
def close_api():
|
||||
nonlocal api_cm, api
|
||||
if api_cm is not None:
|
||||
try:
|
||||
api_cm.__exit__(None, None, None)
|
||||
finally:
|
||||
api_cm = None
|
||||
api = None
|
||||
|
||||
open_api()
|
||||
try:
|
||||
for index, (code, ranges) in enumerate(pending, start=1):
|
||||
try:
|
||||
rows: list[dict] = []
|
||||
for gap_start, gap_end in ranges:
|
||||
if api is None:
|
||||
open_api()
|
||||
rows.extend(
|
||||
_fetch_one_stock_with_api_retry(
|
||||
api,
|
||||
code,
|
||||
gap_start.replace("-", ""),
|
||||
gap_end.replace("-", ""),
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
)
|
||||
)
|
||||
if rows:
|
||||
write_buffer.extend(rows)
|
||||
success += 1
|
||||
total_rows += len(rows)
|
||||
if index % progress_every == 0 or index == total:
|
||||
elapsed = time.time() - t_start
|
||||
avg = elapsed / index
|
||||
eta = avg * (total - index)
|
||||
_logger.info(
|
||||
"[%d/%d] 日线进度，成功:%d 失败:%d 累计获取:%d 行 待写入:%d 剩余:%.0fs",
|
||||
index,
|
||||
total,
|
||||
success,
|
||||
fail,
|
||||
total_rows,
|
||||
len(write_buffer),
|
||||
eta,
|
||||
)
|
||||
if len(write_buffer) >= write_batch_size:
|
||||
flush_rows("batch")
|
||||
else:
|
||||
fail += 1
|
||||
failed_codes.append(code)
|
||||
_logger.warning("日线 %s 没有返回可写入数据，缺口 %d 段", code, len(ranges))
|
||||
except KeyboardInterrupt:
|
||||
flush_rows("interrupt")
|
||||
_logger.warning("日线抓取被用户中断，进度 %d/%d，成功:%d 失败:%d", index - 1, total, success, fail)
|
||||
return
|
||||
except Exception as exc:
|
||||
if _is_tdx_session_error(exc):
|
||||
flush_rows("reconnect")
|
||||
close_api()
|
||||
_logger.warning("日线 %s 连接异常，已重连后留待下次补抓: %s", code, exc)
|
||||
else:
|
||||
fail += 1
|
||||
failed_codes.append(code)
|
||||
_logger.warning("日线 %s 获取失败: %s", code, exc)
|
||||
|
||||
time.sleep(delay)
|
||||
finally:
|
||||
close_api()
|
||||
|
||||
flush_rows("finish")
|
||||
if failed_codes:
|
||||
_logger.warning("日线失败股票 %d 只: %s", len(failed_codes), _preview_codes(failed_codes))
|
||||
_logger.info(
|
||||
"日线抓取完成，成功:%d 失败:%d 累计写入:%d 行 总耗时:%.1fs",
|
||||
success,
|
||||
fail,
|
||||
total_rows,
|
||||
time.time() - t_start,
|
||||
)
|
||||
|
||||
__all__ = [
|
||||
"TdxUnavailableError",
|
||||
"bootstrap_stock_list",
|
||||
"connected_session",
|
||||
"fetch_one_stock",
|
||||
"fetch_one_stock_with_retry",
|
||||
"fetch_stock_list_rows",
|
||||
"fetch_tdx_daily",
|
||||
"fetch_trading_days",
|
||||
"infer_tdx_market",
|
||||
"_tdx_bars_to_rows",
|
||||
]
|
||||
@@ -0,0 +1,726 @@
|
||||
"""分钟 K 数据抓取。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import IndexMin5, TradingDay, batch_upsert, get_session, get_stock_codes, get_stock_minute_model, normalize_minute_freq
|
||||
from src.db.status import get_suspended_trade_days
|
||||
from src.fetchers import tdx_daily
|
||||
from src.fetchers.tdx_minute_utils import (
|
||||
_build_missing_trading_day_ranges,
|
||||
_config_float,
|
||||
_config_int,
|
||||
_covered_days_from_ranges,
|
||||
_format_ranges_for_log,
|
||||
_get_existing_minute_days,
|
||||
_get_latest_minute_dates,
|
||||
_is_completed_trading_day,
|
||||
_missing_days_from_rows,
|
||||
_normalize_db_code,
|
||||
_parse_symbol_list,
|
||||
_preview_codes,
|
||||
_split_supported_min5_codes,
|
||||
)
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import select
|
||||
|
||||
_logger = get_logger("minute")
|
||||
|
||||
TDX_MINUTE_CATEGORIES = {
|
||||
5: 0,
|
||||
}
|
||||
TDX_MINUTE_EXPECTED_BARS_PER_DAY = {
|
||||
5: 48,
|
||||
}
|
||||
TDX_MINUTE_DEFAULT_MAX_PAGES = 250
|
||||
MARKET_INDEX_MIN5 = (
|
||||
("000001", "tdx", 1, "上证指数"),
|
||||
("000016", "tdx", 1, "上证50"),
|
||||
("000300", "tdx", 1, "沪深300"),
|
||||
("000905", "tdx", 1, "中证500"),
|
||||
("000852", "tdx", 1, "中证1000"),
|
||||
("932000", "eastmoney", None, "中证2000"),
|
||||
("000688", "tdx", 1, "科创50"),
|
||||
("399006", "tdx", 0, "创业板指"),
|
||||
)
|
||||
EASTMONEY_INDEX_SECIDS = {"932000": "2.932000"}
|
||||
|
||||
|
||||
def _tdx_minute_max_pages() -> int:
|
||||
try:
|
||||
cfg = get_fetch_config()
|
||||
return max(int(cfg.get("minute_max_pages", cfg.get("tdx_minute_max_pages", TDX_MINUTE_DEFAULT_MAX_PAGES))), 1)
|
||||
except Exception:
|
||||
return TDX_MINUTE_DEFAULT_MAX_PAGES
|
||||
|
||||
|
||||
def minute_frequency_to_category(freq: int | str) -> int:
|
||||
normalized = normalize_minute_freq(freq)
|
||||
return TDX_MINUTE_CATEGORIES[normalized]
|
||||
|
||||
|
||||
def _bar_datetime(bar: dict) -> datetime | None:
|
||||
value = bar.get("datetime") or bar.get("date")
|
||||
if value is not None:
|
||||
parsed = pd.to_datetime(value, errors="coerce")
|
||||
if not pd.isna(parsed):
|
||||
return parsed.to_pydatetime()
|
||||
|
||||
year = bar.get("year")
|
||||
month = bar.get("month")
|
||||
day = bar.get("day")
|
||||
hour = bar.get("hour", 0)
|
||||
minute = bar.get("minute", 0)
|
||||
if year and month and day:
|
||||
try:
|
||||
return datetime(int(year), int(month), int(day), int(hour or 0), int(minute or 0))
|
||||
except (TypeError, ValueError):
|
||||
return None
|
||||
return None
|
||||
|
||||
|
||||
def _tdx_minute_bars_to_rows(
|
||||
code: str,
|
||||
bars: list[dict],
|
||||
*,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
) -> list[dict]:
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]} 00:00:00"
|
||||
end_text = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]} 23:59:59"
|
||||
rows_by_dt: dict[datetime, dict] = {}
|
||||
for bar in bars:
|
||||
dt_value = _bar_datetime(bar)
|
||||
if dt_value is None:
|
||||
continue
|
||||
dt_text = dt_value.strftime("%Y-%m-%d %H:%M:%S")
|
||||
if dt_text < start_text or dt_text > end_text:
|
||||
continue
|
||||
row = {
|
||||
"code": _normalize_db_code(code),
|
||||
"datetime": dt_value,
|
||||
"open": tdx_daily._to_float(bar.get("open")),
|
||||
"high": tdx_daily._to_float(bar.get("high")),
|
||||
"low": tdx_daily._to_float(bar.get("low")),
|
||||
"close": tdx_daily._to_float(bar.get("close")),
|
||||
"volume": tdx_daily._to_float(bar.get("vol", bar.get("volume"))),
|
||||
"amount": tdx_daily._to_float(bar.get("amount")),
|
||||
}
|
||||
if all(row.get(field) is not None for field in ("open", "high", "low", "close", "volume", "amount")):
|
||||
rows_by_dt[dt_value] = row
|
||||
|
||||
return [rows_by_dt[key] for key in sorted(rows_by_dt)]
|
||||
|
||||
|
||||
def _fetch_bars_payload(api, market: int, code: str, *, freq: int, start_date: str) -> list[dict]:
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
category = minute_frequency_to_category(freq)
|
||||
rows: list[dict] = []
|
||||
for page in range(_tdx_minute_max_pages()):
|
||||
offset = page * tdx_daily.TDX_PAGE_SIZE
|
||||
payload = api.get_security_bars(category, market, code, offset, tdx_daily.TDX_PAGE_SIZE)
|
||||
if not payload:
|
||||
break
|
||||
rows.extend(payload)
|
||||
dates = [
|
||||
dt_value.strftime("%Y-%m-%d")
|
||||
for dt_value in (_bar_datetime(item) for item in payload)
|
||||
if dt_value is not None
|
||||
]
|
||||
if dates and min(dates) <= start_text:
|
||||
break
|
||||
return rows
|
||||
|
||||
|
||||
def _fetch_index_bars_payload(api, market: int, code: str, *, freq: int, start_date: str) -> list[dict]:
|
||||
start_text = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
category = minute_frequency_to_category(freq)
|
||||
rows: list[dict] = []
|
||||
for page in range(_tdx_minute_max_pages()):
|
||||
offset = page * tdx_daily.TDX_PAGE_SIZE
|
||||
payload = api.get_index_bars(category, market, code, offset, tdx_daily.TDX_PAGE_SIZE)
|
||||
if not payload:
|
||||
break
|
||||
rows.extend(payload)
|
||||
dates = [
|
||||
dt_value.strftime("%Y-%m-%d")
|
||||
for dt_value in (_bar_datetime(item) for item in payload)
|
||||
if dt_value is not None
|
||||
]
|
||||
if dates and min(dates) <= start_text:
|
||||
break
|
||||
return rows
|
||||
|
||||
|
||||
def _fetch_eastmoney_index_min5_bars(code: str, *, attempts: int = 3) -> list[dict]:
|
||||
"""获取 TDX 未覆盖指数的一分钟分时，并聚合为 5 分钟 K 线。"""
|
||||
import requests
|
||||
|
||||
url = "https://push2.eastmoney.com/api/qt/stock/trends2/get"
|
||||
params = {
|
||||
"secid": EASTMONEY_INDEX_SECIDS.get(code, f"2.{code}"),
|
||||
"ndays": "5",
|
||||
"iscr": "0",
|
||||
"iscca": "0",
|
||||
"fields1": "f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13",
|
||||
"fields2": "f51,f52,f53,f54,f55,f56,f57,f58",
|
||||
}
|
||||
headers = {
|
||||
"Accept": "application/json, text/plain, */*",
|
||||
"Referer": "https://quote.eastmoney.com/",
|
||||
"User-Agent": "Mozilla/5.0",
|
||||
}
|
||||
last_error: Exception | None = None
|
||||
for attempt in range(1, max(attempts, 1) + 1):
|
||||
try:
|
||||
response = requests.get(url, params=params, headers=headers, timeout=15)
|
||||
response.raise_for_status()
|
||||
payload = response.json()
|
||||
data = payload.get("data") or {}
|
||||
if str(data.get("code", code)) != code:
|
||||
raise ValueError(f"返回指数代码异常: {data.get('code')}")
|
||||
return _eastmoney_trends_to_min5_bars(data.get("trends") or [])
|
||||
except Exception as exc:
|
||||
last_error = exc
|
||||
if attempt < max(attempts, 1):
|
||||
time.sleep(0.5 * attempt)
|
||||
assert last_error is not None
|
||||
raise RuntimeError(f"东方财富指数 {code} 分钟数据获取失败: {last_error}") from last_error
|
||||
|
||||
|
||||
def _eastmoney_trends_to_min5_bars(trends: list[str]) -> list[dict]:
|
||||
"""将交易时段内的一分钟分时聚合为标准 5 分钟 K 线。"""
|
||||
buckets: dict[datetime, list[list[str]]] = {}
|
||||
for item in trends:
|
||||
values = item.split(",")
|
||||
if len(values) < 7:
|
||||
continue
|
||||
dt_value = pd.to_datetime(values[0], errors="coerce")
|
||||
if pd.isna(dt_value):
|
||||
continue
|
||||
minute_of_day = dt_value.hour * 60 + dt_value.minute
|
||||
if 9 * 60 + 31 <= minute_of_day <= 11 * 60 + 30:
|
||||
session_start = 9 * 60 + 31
|
||||
elif 13 * 60 + 1 <= minute_of_day <= 15 * 60:
|
||||
session_start = 13 * 60 + 1
|
||||
else:
|
||||
continue
|
||||
bucket_end_minute = session_start + ((minute_of_day - session_start) // 5 + 1) * 5 - 1
|
||||
bucket_end = dt_value.replace(
|
||||
hour=bucket_end_minute // 60,
|
||||
minute=bucket_end_minute % 60,
|
||||
second=0,
|
||||
microsecond=0,
|
||||
).to_pydatetime()
|
||||
buckets.setdefault(bucket_end, []).append(values)
|
||||
|
||||
bars: list[dict] = []
|
||||
for bucket_end in sorted(buckets):
|
||||
values_list = buckets[bucket_end]
|
||||
if len(values_list) < 5:
|
||||
continue
|
||||
opens = [tdx_daily._to_float(values[1]) for values in values_list]
|
||||
closes = [tdx_daily._to_float(values[2]) for values in values_list]
|
||||
highs = [tdx_daily._to_float(values[3]) for values in values_list]
|
||||
lows = [tdx_daily._to_float(values[4]) for values in values_list]
|
||||
volumes = [tdx_daily._to_float(values[5]) for values in values_list]
|
||||
amounts = [tdx_daily._to_float(values[6]) for values in values_list]
|
||||
numeric_values = opens + closes + highs + lows + volumes + amounts
|
||||
if any(value is None for value in numeric_values):
|
||||
continue
|
||||
bars.append({
|
||||
"datetime": bucket_end,
|
||||
"open": opens[0],
|
||||
"close": closes[-1],
|
||||
"high": max(highs),
|
||||
"low": min(lows),
|
||||
"vol": sum(volumes),
|
||||
"amount": sum(amounts),
|
||||
})
|
||||
return bars
|
||||
|
||||
|
||||
def fetch_one_stock_minute(code: str, start_date: str, end_date: str, *, freq: int | str) -> list[dict]:
|
||||
normalized_freq = normalize_minute_freq(freq)
|
||||
db_code = _normalize_db_code(code)
|
||||
market = tdx_daily.infer_tdx_market(db_code)
|
||||
with tdx_daily.connected_session() as api:
|
||||
bars = _fetch_bars_payload(api, market, db_code, freq=normalized_freq, start_date=start_date)
|
||||
return _tdx_minute_bars_to_rows(db_code, bars, start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
def fetch_index_min5(
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
code: str,
|
||||
source: str,
|
||||
market: int | None,
|
||||
name: str,
|
||||
trading_days: list[str] | None = None,
|
||||
) -> int:
|
||||
"""补齐指定指数 5 分钟行情，供成交额预测页叠加展示。"""
|
||||
target_days = trading_days or _load_trading_days(start_date, end_date)
|
||||
if not target_days:
|
||||
return 0
|
||||
|
||||
end_boundary = (datetime.strptime(end_date, "%Y%m%d") + timedelta(days=1)).strftime("%Y-%m-%d")
|
||||
session = get_session()
|
||||
try:
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(IndexMin5.datetime)
|
||||
.where(IndexMin5.code == code)
|
||||
.where(IndexMin5.datetime >= f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}")
|
||||
.where(IndexMin5.datetime < end_boundary)
|
||||
)
|
||||
existing_counts: dict[str, int] = {}
|
||||
for row in rows:
|
||||
if row[0] is None:
|
||||
continue
|
||||
day = row[0].strftime("%Y-%m-%d")
|
||||
existing_counts[day] = existing_counts.get(day, 0) + 1
|
||||
existing_days = {
|
||||
day for day, count in existing_counts.items()
|
||||
if count >= TDX_MINUTE_EXPECTED_BARS_PER_DAY[5]
|
||||
}
|
||||
except Exception as exc:
|
||||
_logger.warning("%s 分钟表未就绪，跳过本次同步: %s", name, exc)
|
||||
return 0
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
missing_days = [day for day in target_days if day not in existing_days]
|
||||
if not missing_days:
|
||||
return 0
|
||||
|
||||
fetch_start = missing_days[0].replace("-", "")
|
||||
fetch_end = missing_days[-1].replace("-", "")
|
||||
try:
|
||||
if source == "tdx":
|
||||
if market is None:
|
||||
raise ValueError(f"{name} 缺少 TDX 市场标识")
|
||||
with tdx_daily.connected_session() as api:
|
||||
bars = _fetch_index_bars_payload(
|
||||
api,
|
||||
market,
|
||||
code,
|
||||
freq=5,
|
||||
start_date=fetch_start,
|
||||
)
|
||||
elif source == "eastmoney":
|
||||
bars = _fetch_eastmoney_index_min5_bars(code)
|
||||
else:
|
||||
raise ValueError(f"不支持的指数分钟数据源: {source}")
|
||||
except Exception as exc:
|
||||
_logger.warning("%s 5 分钟数据同步失败: %s", name, exc)
|
||||
return 0
|
||||
|
||||
index_rows = _tdx_minute_bars_to_rows(
|
||||
code,
|
||||
bars,
|
||||
start_date=fetch_start,
|
||||
end_date=fetch_end,
|
||||
)
|
||||
if not index_rows:
|
||||
_logger.warning("%s 5 分钟数据未返回可写入记录", name)
|
||||
return 0
|
||||
batch_upsert(IndexMin5, index_rows, ["code", "datetime"])
|
||||
_logger.info("%s 5 分钟数据写入 %d 行", name, len(index_rows))
|
||||
return len(index_rows)
|
||||
|
||||
|
||||
def fetch_market_indices_min5(
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
trading_days: list[str] | None = None,
|
||||
) -> int:
|
||||
"""补齐成交额预测使用的大盘、小盘指数分钟行情。"""
|
||||
return sum(
|
||||
fetch_index_min5(
|
||||
start_date,
|
||||
end_date,
|
||||
code=code,
|
||||
source=source,
|
||||
market=market,
|
||||
name=name,
|
||||
trading_days=trading_days,
|
||||
)
|
||||
for code, source, market, name in MARKET_INDEX_MIN5
|
||||
)
|
||||
|
||||
|
||||
def _fetch_one_stock_minute_with_api(
|
||||
api,
|
||||
code: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
freq: int | str,
|
||||
) -> list[dict]:
|
||||
normalized_freq = normalize_minute_freq(freq)
|
||||
db_code = _normalize_db_code(code)
|
||||
market = tdx_daily.infer_tdx_market(db_code)
|
||||
bars = _fetch_bars_payload(api, market, db_code, freq=normalized_freq, start_date=start_date)
|
||||
return _tdx_minute_bars_to_rows(db_code, bars, start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
def _fetch_one_stock_minute_with_api_retry(
|
||||
api,
|
||||
code: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
freq: int | str,
|
||||
retry: int = 3,
|
||||
delay: float = 0.1,
|
||||
) -> list[dict]:
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
return _fetch_one_stock_minute_with_api(api, code, start_date, end_date, freq=freq)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
raise
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
_logger.warning("%sm 分钟 %s 第 %d 次失败，%.1fs 后重试: %s", freq, code, attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
if last_exc is not None:
|
||||
raise last_exc
|
||||
return []
|
||||
|
||||
|
||||
def fetch_one_stock_minute_with_retry(
|
||||
code: str,
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
freq: int | str,
|
||||
retry: int = 3,
|
||||
delay: float = 0.1,
|
||||
) -> list[dict]:
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
return fetch_one_stock_minute(code, start_date, end_date, freq=freq)
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
raise
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
_logger.warning("%sm 分钟 %s 第 %d 次失败，%.1fs 后重试: %s", freq, code, attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
if last_exc is not None:
|
||||
raise last_exc
|
||||
return []
|
||||
|
||||
|
||||
def _minute_complete_bars(cfg: dict, freq: int) -> int:
|
||||
key = f"min{freq}_complete_min_bars"
|
||||
return _config_int(cfg, key, TDX_MINUTE_EXPECTED_BARS_PER_DAY[freq])
|
||||
|
||||
|
||||
def _tdx_minute_delay(cfg: dict) -> float:
|
||||
if "minute_delay" in cfg:
|
||||
return _config_float(cfg, "minute_delay", 0.8, min_value=0.0)
|
||||
return _config_float(cfg, "tdx_minute_delay", 0.8, min_value=0.0)
|
||||
|
||||
|
||||
def _resolve_symbols(symbol: str | None) -> list[str]:
|
||||
explicit_symbols = _parse_symbol_list(symbol)
|
||||
if explicit_symbols:
|
||||
return explicit_symbols
|
||||
symbols = get_stock_codes()
|
||||
if symbols:
|
||||
return symbols
|
||||
_logger.info("股票列表为空，先补齐股票列表后再下载分钟数据")
|
||||
return tdx_daily.bootstrap_stock_list()
|
||||
|
||||
|
||||
def _load_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
sd = f"{start_date[:4]}-{start_date[4:6]}-{start_date[6:8]}"
|
||||
ed = f"{end_date[:4]}-{end_date[4:6]}-{end_date[6:8]}"
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
)
|
||||
days = [str(row[0]) for row in result]
|
||||
finally:
|
||||
session.close()
|
||||
if days:
|
||||
return days
|
||||
_logger.info("本地交易日历为空，改用指数日线推导交易日")
|
||||
return tdx_daily.fetch_trading_days(start_date, end_date)
|
||||
|
||||
|
||||
def _fetch_one_freq(
|
||||
*,
|
||||
freq: int,
|
||||
symbols: list[str],
|
||||
trading_days: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
retry: int,
|
||||
delay: float,
|
||||
progress_every: int,
|
||||
empty_breaker_limit: int,
|
||||
now: datetime | None,
|
||||
cfg: dict,
|
||||
limit: int | None,
|
||||
suspended_days_by_code: dict[str, set[str]],
|
||||
) -> None:
|
||||
model_cls = get_stock_minute_model(freq)
|
||||
complete_min_bars = _minute_complete_bars(cfg, freq)
|
||||
existing_days_by_code = _get_existing_minute_days(
|
||||
symbols,
|
||||
start_date,
|
||||
end_date,
|
||||
min_bars=complete_min_bars,
|
||||
freq=freq,
|
||||
)
|
||||
latest_dates = _get_latest_minute_dates(symbols, freq=freq)
|
||||
|
||||
pending_symbols: list[tuple[str, list[tuple[str, str]]]] = []
|
||||
skipped_up_to_date = 0
|
||||
suspended_day_count = 0
|
||||
for raw_symbol in symbols:
|
||||
db_code = _normalize_db_code(raw_symbol)
|
||||
existing_days = existing_days_by_code.get(db_code, set())
|
||||
suspended_days = suspended_days_by_code.get(db_code, set())
|
||||
suspended_day_count += len((set(trading_days) - existing_days) & suspended_days)
|
||||
missing_ranges = _build_missing_trading_day_ranges(trading_days, existing_days | suspended_days)
|
||||
if not missing_ranges:
|
||||
skipped_up_to_date += 1
|
||||
continue
|
||||
pending_symbols.append((db_code, missing_ranges))
|
||||
if limit is not None and len(pending_symbols) >= limit:
|
||||
break
|
||||
|
||||
_logger.info(
|
||||
"%sm 分钟写入表 %s，区间 %s ~ %s，股票数:%d，待抓取:%d，已最新:%d，停牌标记跳过:%d日，已有最新日期股票:%d，limit:%s",
|
||||
freq,
|
||||
model_cls.__tablename__,
|
||||
start_date,
|
||||
end_date,
|
||||
len(symbols),
|
||||
len(pending_symbols),
|
||||
skipped_up_to_date,
|
||||
suspended_day_count,
|
||||
len(latest_dates),
|
||||
limit if limit is not None else "all",
|
||||
)
|
||||
if not pending_symbols:
|
||||
return
|
||||
|
||||
success = 0
|
||||
fail = 0
|
||||
total_rows = 0
|
||||
failed_codes: list[str] = []
|
||||
consecutive_no_write = 0
|
||||
t_start = time.time()
|
||||
total_symbols = len(pending_symbols)
|
||||
|
||||
with tdx_daily.connected_session() as api:
|
||||
for index, (db_code, missing_ranges) in enumerate(pending_symbols, start=1):
|
||||
range_text = _format_ranges_for_log(missing_ranges)
|
||||
fetch_start = missing_ranges[0][0].replace("-", "")
|
||||
fetch_end = missing_ranges[-1][1].replace("-", "")
|
||||
t_item = time.time()
|
||||
remaining = total_symbols - index
|
||||
try:
|
||||
fetched_rows = _fetch_one_stock_minute_with_api_retry(
|
||||
api,
|
||||
db_code,
|
||||
fetch_start,
|
||||
fetch_end,
|
||||
freq=freq,
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
)
|
||||
symbol_rows: list[dict] = []
|
||||
covered_days = _covered_days_from_ranges(trading_days, missing_ranges)
|
||||
for row in fetched_rows:
|
||||
day = row["datetime"].strftime("%Y-%m-%d")
|
||||
if day in covered_days:
|
||||
symbol_rows.append(row)
|
||||
|
||||
missing_completed_days = [
|
||||
day
|
||||
for day in _missing_days_from_rows(trading_days, missing_ranges, symbol_rows)
|
||||
if _is_completed_trading_day(day, now=now)
|
||||
]
|
||||
|
||||
if symbol_rows:
|
||||
batch_upsert(model_cls, symbol_rows, ["code", "datetime"])
|
||||
success += 1
|
||||
total_rows += len(symbol_rows)
|
||||
consecutive_no_write = 0
|
||||
else:
|
||||
fail += 1
|
||||
consecutive_no_write += 1
|
||||
_logger.warning(
|
||||
"%sm %s 没有返回可写入数据，缺少已收盘交易日 %d 天，缺口 %d 段 %s，进度 %d/%d",
|
||||
freq,
|
||||
db_code,
|
||||
len(missing_completed_days),
|
||||
len(missing_ranges),
|
||||
range_text,
|
||||
index,
|
||||
total_symbols,
|
||||
)
|
||||
except KeyboardInterrupt:
|
||||
_logger.warning(
|
||||
"%sm 分钟抓取被用户中断，进度 %d/%d，成功:%d 失败:%d 累计写入:%d 行",
|
||||
freq,
|
||||
index - 1,
|
||||
total_symbols,
|
||||
success,
|
||||
fail,
|
||||
total_rows,
|
||||
)
|
||||
return
|
||||
except Exception as exc:
|
||||
fail += 1
|
||||
consecutive_no_write += 1
|
||||
failed_codes.append(db_code)
|
||||
_logger.warning(
|
||||
"%sm %s 获取失败: %s，累计写入 %d 行，缺口 %d 段 %s，进度 %d/%d，剩余 %d 只",
|
||||
freq,
|
||||
db_code,
|
||||
exc,
|
||||
total_rows,
|
||||
len(missing_ranges),
|
||||
range_text,
|
||||
index,
|
||||
total_symbols,
|
||||
remaining,
|
||||
)
|
||||
|
||||
if consecutive_no_write >= empty_breaker_limit:
|
||||
_logger.warning(
|
||||
"%sm 分钟连续 %d 只股票没有写入，停止该周期抓取；请检查服务器/网络后重跑，已写入:%d 行",
|
||||
freq,
|
||||
consecutive_no_write,
|
||||
total_rows,
|
||||
)
|
||||
break
|
||||
|
||||
if index % progress_every == 0 or index == total_symbols:
|
||||
elapsed = time.time() - t_start
|
||||
avg = elapsed / index
|
||||
eta = avg * (total_symbols - index)
|
||||
_logger.info(
|
||||
"[%d/%d] %sm 分钟进度：成功:%d 失败:%d 累计写入:%d 行 本只耗时:%.1fs 剩余:%.0fs",
|
||||
index,
|
||||
total_symbols,
|
||||
freq,
|
||||
success,
|
||||
fail,
|
||||
total_rows,
|
||||
time.time() - t_item,
|
||||
eta,
|
||||
)
|
||||
time.sleep(delay)
|
||||
|
||||
if failed_codes:
|
||||
_logger.warning("%sm 分钟失败股票 %d 只: %s", freq, len(failed_codes), _preview_codes(failed_codes))
|
||||
|
||||
_logger.info(
|
||||
"%sm 分钟抓取完成，成功:%d 失败:%d 累计写入:%d 行 总耗时:%.1fs",
|
||||
freq,
|
||||
success,
|
||||
fail,
|
||||
total_rows,
|
||||
time.time() - t_start,
|
||||
)
|
||||
|
||||
|
||||
def fetch_tdx_minute(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
limit: int | None = None,
|
||||
now: datetime | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
"""抓取股票和常用指数的 5 分钟 K。"""
|
||||
cfg = get_fetch_config()
|
||||
delay = _tdx_minute_delay(cfg)
|
||||
retry = _config_int(cfg, "retry", 3)
|
||||
progress_every = _config_int(cfg, "progress_every", 50)
|
||||
empty_breaker_limit = _config_int(cfg, "empty_breaker_limit", 20)
|
||||
if limit is not None:
|
||||
limit = max(int(limit), 1)
|
||||
|
||||
if start_date is None:
|
||||
start_date = "20150101"
|
||||
if end_date is None:
|
||||
end_date = (now or datetime.now()).strftime("%Y%m%d")
|
||||
|
||||
symbols = _resolve_symbols(symbol)
|
||||
original_symbol_count = len(symbols)
|
||||
symbols, skipped_symbols = _split_supported_min5_codes(symbols)
|
||||
if skipped_symbols:
|
||||
_logger.info("分钟数据跳过不支持代码 %d 只: %s", len(skipped_symbols), _preview_codes(skipped_symbols))
|
||||
if not symbols:
|
||||
_logger.warning("分钟数据没有可抓取股票，原始股票数 %d", original_symbol_count)
|
||||
return
|
||||
|
||||
trading_days = _load_trading_days(start_date, end_date)
|
||||
if not trading_days:
|
||||
_logger.warning("区间内没有交易日，分钟数据无需抓取")
|
||||
return
|
||||
|
||||
suspended_days_by_code = get_suspended_trade_days(symbols, start_date, end_date) if skip_suspended else {}
|
||||
|
||||
_logger.info(
|
||||
"5分钟数据开始，区间:%s ~ %s，股票:%d",
|
||||
start_date,
|
||||
end_date,
|
||||
len(symbols),
|
||||
)
|
||||
_fetch_one_freq(
|
||||
freq=5,
|
||||
symbols=symbols,
|
||||
trading_days=trading_days,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
retry=retry,
|
||||
delay=delay,
|
||||
progress_every=progress_every,
|
||||
empty_breaker_limit=empty_breaker_limit,
|
||||
now=now,
|
||||
cfg=cfg,
|
||||
limit=limit,
|
||||
suspended_days_by_code=suspended_days_by_code,
|
||||
)
|
||||
fetch_market_indices_min5(start_date, end_date, trading_days=trading_days)
|
||||
_logger.info("5分钟数据抓取完成")
|
||||
|
||||
|
||||
__all__ = [
|
||||
"TDX_MINUTE_CATEGORIES",
|
||||
"fetch_one_stock_minute",
|
||||
"fetch_one_stock_minute_with_retry",
|
||||
"fetch_index_min5",
|
||||
"fetch_market_indices_min5",
|
||||
"fetch_tdx_minute",
|
||||
"minute_frequency_to_category",
|
||||
"_tdx_minute_bars_to_rows",
|
||||
]
|
||||
@@ -0,0 +1,283 @@
|
||||
"""分钟数据缺口分析与代码处理工具。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, time as datetime_time, timedelta
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.db import get_session, get_stock_minute_model
|
||||
|
||||
DB_QUERY_BATCH_SIZE = 500
|
||||
MINUTE_DEFAULT_COMPLETE_AFTER = datetime_time(15, 10)
|
||||
|
||||
|
||||
def _normalize_db_code(symbol: str) -> str:
|
||||
code = (symbol or "").strip()
|
||||
if not code:
|
||||
return code
|
||||
|
||||
lower = code.lower()
|
||||
if lower.startswith(("sh.", "sz.")) and len(lower) > 3:
|
||||
return lower[3:]
|
||||
if lower.startswith(("sh", "sz")) and len(lower) > 2:
|
||||
return lower[2:].lstrip(".")
|
||||
|
||||
upper = code.upper()
|
||||
if "." in upper:
|
||||
base, suffix = upper.split(".", 1)
|
||||
if suffix in {"SH", "SZ"}:
|
||||
return base
|
||||
return code
|
||||
|
||||
|
||||
def _is_supported_minute_code(symbol: str) -> bool:
|
||||
db_code = _normalize_db_code(symbol)
|
||||
return (
|
||||
len(db_code) == 6
|
||||
and db_code.isdigit()
|
||||
and (db_code.startswith("6") or db_code.startswith("0") or db_code.startswith("30"))
|
||||
)
|
||||
|
||||
|
||||
def _split_supported_min5_codes(symbols: list[str]) -> tuple[list[str], list[str]]:
|
||||
supported: list[str] = []
|
||||
skipped: list[str] = []
|
||||
for symbol in symbols:
|
||||
db_code = _normalize_db_code(symbol)
|
||||
if _is_supported_minute_code(db_code):
|
||||
supported.append(db_code)
|
||||
else:
|
||||
skipped.append(db_code)
|
||||
return list(dict.fromkeys(supported)), list(dict.fromkeys(skipped))
|
||||
|
||||
|
||||
def _parse_symbol_list(symbol: str | None) -> list[str]:
|
||||
if not symbol:
|
||||
return []
|
||||
return [
|
||||
_normalize_db_code(item)
|
||||
for item in symbol.replace("，", ",").split(",")
|
||||
if _normalize_db_code(item)
|
||||
]
|
||||
|
||||
|
||||
def _parse_yyyymmdd(value: str) -> date:
|
||||
return datetime.strptime(value, "%Y%m%d").date()
|
||||
|
||||
|
||||
def _coerce_date(value) -> date | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, datetime):
|
||||
return value.date()
|
||||
if isinstance(value, date):
|
||||
return value
|
||||
text = str(value).strip()
|
||||
if not text:
|
||||
return None
|
||||
for fmt in ("%Y-%m-%d", "%Y%m%d"):
|
||||
try:
|
||||
return datetime.strptime(text[:10] if fmt == "%Y-%m-%d" else text[:8], fmt).date()
|
||||
except ValueError:
|
||||
continue
|
||||
return None
|
||||
|
||||
|
||||
def _stock_code_variants(code: str) -> list[str]:
|
||||
db_code = _normalize_db_code(code)
|
||||
if not db_code:
|
||||
return []
|
||||
|
||||
variants = {db_code, db_code.upper(), db_code.lower()}
|
||||
if db_code.isdigit() and len(db_code) == 6:
|
||||
if db_code.startswith("6"):
|
||||
variants.update({f"{db_code}.SH", f"{db_code}.sh", f"sh{db_code}", f"SH{db_code}"})
|
||||
elif db_code.startswith(("0", "3")):
|
||||
variants.update({f"{db_code}.SZ", f"{db_code}.sz", f"sz{db_code}", f"SZ{db_code}"})
|
||||
return sorted(variants)
|
||||
|
||||
|
||||
def _chunked(items: list[str], batch_size: int = DB_QUERY_BATCH_SIZE):
|
||||
for idx in range(0, len(items), batch_size):
|
||||
batch = items[idx: idx + batch_size]
|
||||
if batch:
|
||||
yield batch
|
||||
|
||||
|
||||
def _get_latest_minute_dates(symbols: list[str], *, freq: int = 5) -> dict[str, date]:
|
||||
if not symbols:
|
||||
return {}
|
||||
|
||||
candidate_codes = sorted({
|
||||
variant
|
||||
for symbol in symbols
|
||||
for variant in _stock_code_variants(symbol)
|
||||
})
|
||||
if not candidate_codes:
|
||||
return {}
|
||||
|
||||
model_cls = get_stock_minute_model(freq)
|
||||
session = get_session()
|
||||
try:
|
||||
latest: dict[str, date] = {}
|
||||
for batch in _chunked(candidate_codes):
|
||||
result = session.execute(
|
||||
select(model_cls.code, func.max(model_cls.datetime))
|
||||
.where(model_cls.code.in_(batch))
|
||||
.group_by(model_cls.code)
|
||||
)
|
||||
for code, dt_value in result:
|
||||
if dt_value:
|
||||
normalized_code = _normalize_db_code(code)
|
||||
latest_date = dt_value.date()
|
||||
current = latest.get(normalized_code)
|
||||
if current is None or latest_date > current:
|
||||
latest[normalized_code] = latest_date
|
||||
return latest
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _get_existing_minute_days(
|
||||
symbols: list[str],
|
||||
start_date: str,
|
||||
end_date: str,
|
||||
*,
|
||||
min_bars: int,
|
||||
freq: int = 5,
|
||||
) -> dict[str, set[str]]:
|
||||
if not symbols:
|
||||
return {}
|
||||
|
||||
candidate_codes = sorted({
|
||||
variant
|
||||
for symbol in symbols
|
||||
for variant in _stock_code_variants(symbol)
|
||||
})
|
||||
if not candidate_codes:
|
||||
return {}
|
||||
|
||||
sd = _parse_yyyymmdd(start_date)
|
||||
ed = _parse_yyyymmdd(end_date)
|
||||
next_day = ed + timedelta(days=1)
|
||||
|
||||
model_cls = get_stock_minute_model(freq)
|
||||
session = get_session()
|
||||
try:
|
||||
existing_days: dict[str, set[str]] = {}
|
||||
for batch in _chunked(candidate_codes):
|
||||
day_expr = func.date(model_cls.datetime)
|
||||
result = session.execute(
|
||||
select(model_cls.code, day_expr, func.count())
|
||||
.where(model_cls.code.in_(batch))
|
||||
.where(model_cls.datetime >= datetime.combine(sd, datetime.min.time()))
|
||||
.where(model_cls.datetime < datetime.combine(next_day, datetime.min.time()))
|
||||
.group_by(model_cls.code, day_expr)
|
||||
)
|
||||
for code, day_value, row_count in result:
|
||||
normalized_code = _normalize_db_code(code)
|
||||
day = _coerce_date(day_value)
|
||||
if day is None or int(row_count or 0) < min_bars:
|
||||
continue
|
||||
existing_days.setdefault(normalized_code, set()).add(day.strftime("%Y-%m-%d"))
|
||||
return existing_days
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _build_missing_trading_day_ranges(trading_days: list[str], existing_days: set[str]) -> list[tuple[str, str]]:
|
||||
missing_indexes = [idx for idx, day in enumerate(trading_days) if day not in existing_days]
|
||||
if not missing_indexes:
|
||||
return []
|
||||
|
||||
ranges: list[tuple[str, str]] = []
|
||||
range_start = missing_indexes[0]
|
||||
prev_index = missing_indexes[0]
|
||||
for idx in missing_indexes[1:]:
|
||||
if idx == prev_index + 1:
|
||||
prev_index = idx
|
||||
continue
|
||||
ranges.append((trading_days[range_start], trading_days[prev_index]))
|
||||
range_start = prev_index = idx
|
||||
ranges.append((trading_days[range_start], trading_days[prev_index]))
|
||||
return ranges
|
||||
|
||||
|
||||
def _format_ranges_for_log(ranges: list[tuple[str, str]], limit: int = 3) -> str:
|
||||
if not ranges:
|
||||
return "[]"
|
||||
preview = [
|
||||
start if start == end else f"{start}~{end}"
|
||||
for start, end in ranges[:limit]
|
||||
]
|
||||
if len(ranges) > limit:
|
||||
preview.append(f"...(+{len(ranges) - limit})")
|
||||
return "[" + ", ".join(preview) + "]"
|
||||
|
||||
|
||||
def _preview_codes(codes: list[str], limit: int = 10) -> str:
|
||||
if not codes:
|
||||
return "[]"
|
||||
preview = codes[:limit]
|
||||
suffix = ""
|
||||
if len(codes) > limit:
|
||||
suffix = f", ...(+{len(codes) - limit})"
|
||||
return "[" + ", ".join(preview) + suffix + "]"
|
||||
|
||||
|
||||
def _covered_days_from_ranges(
|
||||
trading_days: list[str],
|
||||
missing_ranges: list[tuple[str, str]],
|
||||
) -> set[str]:
|
||||
return {
|
||||
day
|
||||
for gap_start, gap_end in missing_ranges
|
||||
for day in trading_days
|
||||
if gap_start <= day <= gap_end
|
||||
}
|
||||
|
||||
|
||||
def _row_dates(rows: list[dict]) -> set[str]:
|
||||
row_days: set[str] = set()
|
||||
for row in rows:
|
||||
dt_value = row.get("datetime")
|
||||
if dt_value is not None:
|
||||
row_days.add(dt_value.strftime("%Y-%m-%d"))
|
||||
return row_days
|
||||
|
||||
|
||||
def _missing_days_from_rows(
|
||||
trading_days: list[str],
|
||||
missing_ranges: list[tuple[str, str]],
|
||||
rows: list[dict],
|
||||
) -> list[str]:
|
||||
expected_days = _covered_days_from_ranges(trading_days, missing_ranges)
|
||||
returned_days = _row_dates(rows)
|
||||
return sorted(expected_days - returned_days)
|
||||
|
||||
|
||||
def _is_completed_trading_day(day_text: str, *, now: datetime | None = None) -> bool:
|
||||
current = now or datetime.now()
|
||||
day = datetime.strptime(day_text, "%Y-%m-%d").date()
|
||||
if day < current.date():
|
||||
return True
|
||||
if day == current.date() and current.time() >= MINUTE_DEFAULT_COMPLETE_AFTER:
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
def _config_int(cfg: dict, key: str, default: int, *, min_value: int = 1) -> int:
|
||||
try:
|
||||
value = int(cfg.get(key, default))
|
||||
except (TypeError, ValueError):
|
||||
value = default
|
||||
return max(value, min_value)
|
||||
|
||||
|
||||
def _config_float(cfg: dict, key: str, default: float, *, min_value: float = 0.0) -> float:
|
||||
try:
|
||||
value = float(cfg.get(key, default))
|
||||
except (TypeError, ValueError):
|
||||
value = default
|
||||
return max(value, min_value)
|
||||
@@ -0,0 +1,237 @@
|
||||
"""交易状态归档：区分交易、停牌、未上市和数据源异常。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import date, datetime, timedelta
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import (
|
||||
StockDaily,
|
||||
StockInfo,
|
||||
StockTradeStatus,
|
||||
TradingDay,
|
||||
batch_upsert,
|
||||
get_session,
|
||||
get_stock_minute_model,
|
||||
)
|
||||
from src.log import get_logger
|
||||
|
||||
_logger = get_logger("trade.status")
|
||||
|
||||
STATUS_TRADING = "trading"
|
||||
STATUS_SUSPENDED = "suspended"
|
||||
STATUS_NOT_LISTED = "not_listed"
|
||||
STATUS_SOURCE_ERROR = "source_error"
|
||||
|
||||
|
||||
def _format_date(value: str) -> str:
|
||||
return f"{value[:4]}-{value[4:6]}-{value[6:8]}"
|
||||
|
||||
|
||||
def _is_daily_supported_code(code: str) -> bool:
|
||||
value = str(code or "").strip()
|
||||
return (
|
||||
len(value) == 6
|
||||
and value.isdigit()
|
||||
and (value.startswith("6") or value.startswith("0") or value.startswith("30"))
|
||||
)
|
||||
|
||||
|
||||
def _parse_iso_day(value: str | date | datetime) -> date:
|
||||
if isinstance(value, datetime):
|
||||
return value.date()
|
||||
if isinstance(value, date):
|
||||
return value
|
||||
text = str(value).strip()
|
||||
if len(text) == 8 and text.isdigit():
|
||||
text = _format_date(text)
|
||||
return datetime.strptime(text[:10], "%Y-%m-%d").date()
|
||||
|
||||
|
||||
def _default_end_date() -> str:
|
||||
return datetime.now().strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _default_start_date() -> str:
|
||||
return (datetime.now() - timedelta(days=30)).strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _config_float(cfg: dict, key: str, default: float) -> float:
|
||||
try:
|
||||
return float(cfg.get(key, default))
|
||||
except (TypeError, ValueError):
|
||||
return default
|
||||
|
||||
|
||||
def _load_trading_days(start_date: str, end_date: str) -> list[date]:
|
||||
sd = _parse_iso_day(start_date)
|
||||
ed = _parse_iso_day(end_date)
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
)
|
||||
return [_parse_iso_day(row[0]) for row in rows]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _load_stock_info() -> dict[str, date | None]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(select(StockInfo.code, StockInfo.ipo_date))
|
||||
return {
|
||||
str(code): (_parse_iso_day(ipo_date) if ipo_date else None)
|
||||
for code, ipo_date in rows
|
||||
if _is_daily_supported_code(str(code))
|
||||
}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _load_daily_codes(day: date) -> set[str]:
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(select(StockDaily.code).where(StockDaily.date == day))
|
||||
return {str(row[0]) for row in rows}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _load_minute_counts(day: date, freq: int) -> dict[str, int]:
|
||||
model_cls = get_stock_minute_model(freq)
|
||||
start_dt = datetime.combine(day, datetime.min.time())
|
||||
end_dt = start_dt + timedelta(days=1)
|
||||
session = get_session()
|
||||
try:
|
||||
rows = session.execute(
|
||||
select(model_cls.code, func.count())
|
||||
.where(model_cls.datetime >= start_dt)
|
||||
.where(model_cls.datetime < end_dt)
|
||||
.group_by(model_cls.code)
|
||||
)
|
||||
return {str(code): int(count or 0) for code, count in rows}
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _status_for_code(
|
||||
*,
|
||||
code: str,
|
||||
day: date,
|
||||
ipo_date: date | None,
|
||||
daily_codes: set[str],
|
||||
coverage_ok: bool,
|
||||
) -> tuple[str, str]:
|
||||
if ipo_date is not None and ipo_date > day:
|
||||
return STATUS_NOT_LISTED, "ipo_date_after_trade_day"
|
||||
if not coverage_ok:
|
||||
return STATUS_SOURCE_ERROR, "daily_market_coverage_below_threshold"
|
||||
if code in daily_codes:
|
||||
return STATUS_TRADING, "daily_exists"
|
||||
return STATUS_SUSPENDED, "listed_without_daily_bar"
|
||||
|
||||
|
||||
def _build_trade_status_rows(
|
||||
*,
|
||||
day: date,
|
||||
stocks: dict[str, date | None],
|
||||
daily_codes: set[str],
|
||||
minute_counts: dict[int, dict[str, int]],
|
||||
coverage_threshold: float,
|
||||
checked_at: datetime,
|
||||
) -> list[dict]:
|
||||
listed_codes = {
|
||||
code
|
||||
for code, ipo_date in stocks.items()
|
||||
if ipo_date is None or ipo_date <= day
|
||||
}
|
||||
coverage = (len(daily_codes & listed_codes) / len(listed_codes)) if listed_codes else 0.0
|
||||
coverage_ok = coverage >= coverage_threshold
|
||||
|
||||
rows: list[dict] = []
|
||||
for code in sorted(stocks):
|
||||
status, reason = _status_for_code(
|
||||
code=code,
|
||||
day=day,
|
||||
ipo_date=stocks[code],
|
||||
daily_codes=daily_codes,
|
||||
coverage_ok=coverage_ok,
|
||||
)
|
||||
row = {
|
||||
"code": code,
|
||||
"date": day,
|
||||
"status": status,
|
||||
"source": "market",
|
||||
"daily_exists": 1 if code in daily_codes else 0,
|
||||
"reason": reason,
|
||||
"checked_at": checked_at,
|
||||
}
|
||||
row["min5_count"] = minute_counts.get(5, {}).get(code, 0)
|
||||
rows.append(row)
|
||||
|
||||
_logger.info(
|
||||
"交易状态 %s 覆盖率 %.2f%%，股票:%d，日K:%d，阈值:%.2f%%",
|
||||
day.isoformat(),
|
||||
coverage * 100,
|
||||
len(stocks),
|
||||
len(daily_codes & set(stocks)),
|
||||
coverage_threshold * 100,
|
||||
)
|
||||
return rows
|
||||
|
||||
|
||||
def fetch_trade_status(
|
||||
*,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
) -> None:
|
||||
"""生成交易状态归档，写入 stock_trade_status。"""
|
||||
start_date = start_date or _default_start_date()
|
||||
end_date = end_date or _default_end_date()
|
||||
cfg = get_fetch_config()
|
||||
coverage_threshold = _config_float(cfg, "trade_status_daily_coverage_threshold", 0.8)
|
||||
coverage_threshold = min(max(coverage_threshold, 0.0), 1.0)
|
||||
|
||||
trading_days = _load_trading_days(start_date, end_date)
|
||||
if not trading_days:
|
||||
_logger.warning("交易状态区间内没有交易日：%s ~ %s", start_date, end_date)
|
||||
return
|
||||
|
||||
stocks = _load_stock_info()
|
||||
if not stocks:
|
||||
_logger.warning("交易状态没有股票列表，请先执行 --stock-info")
|
||||
return
|
||||
|
||||
checked_at = datetime.now()
|
||||
total_rows = 0
|
||||
for day in trading_days:
|
||||
daily_codes = _load_daily_codes(day)
|
||||
minute_counts = {5: _load_minute_counts(day, 5)}
|
||||
rows = _build_trade_status_rows(
|
||||
day=day,
|
||||
stocks=stocks,
|
||||
daily_codes=daily_codes,
|
||||
minute_counts=minute_counts,
|
||||
coverage_threshold=coverage_threshold,
|
||||
checked_at=checked_at,
|
||||
)
|
||||
batch_upsert(StockTradeStatus, rows, ["code", "date"])
|
||||
total_rows += len(rows)
|
||||
|
||||
_logger.info("交易状态归档完成，交易日:%d，写入/更新:%d 行", len(trading_days), total_rows)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"STATUS_NOT_LISTED",
|
||||
"STATUS_SOURCE_ERROR",
|
||||
"STATUS_SUSPENDED",
|
||||
"STATUS_TRADING",
|
||||
"fetch_trade_status",
|
||||
"_build_trade_status_rows",
|
||||
]
|
||||
+19
-83
@@ -1,75 +1,29 @@
|
||||
"""交易日历模块 — AKShare。"""
|
||||
"""兼容入口：交易日历统一使用当前行情实现。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
import random
|
||||
|
||||
import pandas as pd
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import TradingDay, batch_upsert, get_session
|
||||
from src.db import TradingDay, get_session
|
||||
from src.fetchers import tdx_daily
|
||||
from src.log import get_logger
|
||||
from sqlalchemy import select
|
||||
|
||||
_logger = get_logger("trading_day")
|
||||
|
||||
try:
|
||||
import akshare as ak
|
||||
except Exception: # pragma: no cover - 运行环境缺少依赖时再报错
|
||||
ak = None
|
||||
|
||||
|
||||
def _format_date(d: str) -> str:
|
||||
return f"{d[:4]}-{d[4:6]}-{d[6:8]}"
|
||||
|
||||
|
||||
def _fetch_akshare_calendar(*, retry: int = 3, delay: float = 0.1) -> list[str]:
|
||||
"""从 AKShare 获取全量交易日历。"""
|
||||
if ak is None:
|
||||
_logger.error("未安装 akshare，无法抓取交易日历")
|
||||
return []
|
||||
|
||||
retry = max(int(retry), 1)
|
||||
delay = max(float(delay), 0.0)
|
||||
last_exc: Exception | None = None
|
||||
|
||||
for attempt in range(1, retry + 1):
|
||||
try:
|
||||
df = ak.tool_trade_date_hist_sina()
|
||||
if df is None or df.empty:
|
||||
return []
|
||||
trade_col = "trade_date" if "trade_date" in df.columns else df.columns[0]
|
||||
series = pd.to_datetime(df[trade_col], errors="coerce").dropna()
|
||||
return [dt.strftime("%Y-%m-%d") for dt in series]
|
||||
except Exception as exc:
|
||||
last_exc = exc
|
||||
if attempt >= retry:
|
||||
break
|
||||
wait_seconds = delay * attempt if delay > 0 else 0.5 * attempt
|
||||
wait_seconds += random.uniform(0, min(wait_seconds * 0.2, 0.5))
|
||||
_logger.warning("AKShare 获取交易日历第 %d 次失败，%.1fs 后重试: %s", attempt, wait_seconds, exc)
|
||||
time.sleep(wait_seconds)
|
||||
|
||||
if last_exc is not None:
|
||||
_logger.warning("AKShare 获取交易日历失败: %s", last_exc)
|
||||
return []
|
||||
|
||||
|
||||
def _fetch_and_save(start_date: str, end_date: str) -> list[str]:
|
||||
"""从 AKShare 获取交易日并保存到本地表。"""
|
||||
cfg = get_fetch_config()
|
||||
days = _fetch_akshare_calendar(retry=cfg.get("retry", 3), delay=cfg.get("delay", 0.1))
|
||||
if not days:
|
||||
return []
|
||||
|
||||
rows = [{"date": d} for d in days]
|
||||
batch_upsert(TradingDay, rows, ["date"])
|
||||
filtered = [d for d in days if start_date <= d.replace("-", "") <= end_date]
|
||||
_logger.info("交易日历已保存，%d 个交易日（本次范围 %d 天）", len(rows), len(filtered))
|
||||
return filtered
|
||||
def _calendar_covers_range(max_loaded_day, end_date: str) -> bool:
|
||||
if max_loaded_day is None:
|
||||
return False
|
||||
return str(max_loaded_day) >= _format_date(end_date)
|
||||
|
||||
|
||||
def get_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
"""获取指定范围内的交易日列表。"""
|
||||
sd = _format_date(start_date)
|
||||
ed = _format_date(end_date)
|
||||
|
||||
@@ -82,45 +36,27 @@ def get_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
.order_by(TradingDay.date)
|
||||
)
|
||||
days = [str(row[0]) for row in result]
|
||||
|
||||
if days and days[0] <= sd and days[-1] >= ed:
|
||||
max_loaded_day = session.execute(select(func.max(TradingDay.date))).scalar()
|
||||
if _calendar_covers_range(max_loaded_day, end_date):
|
||||
return days
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
fetched = _fetch_and_save(start_date, end_date)
|
||||
fetched = tdx_daily.fetch_trading_days(start_date, end_date)
|
||||
if not fetched:
|
||||
return days if days else []
|
||||
return days
|
||||
return sorted(set(days + fetched))
|
||||
|
||||
|
||||
def fetch_trading_days(start_date: str | None = None, end_date: str | None = None):
|
||||
"""独立抓取交易日历并保存。"""
|
||||
if end_date is None:
|
||||
end_date = time.strftime("%Y%m%d")
|
||||
if start_date is None:
|
||||
start_date = "19901219"
|
||||
days = tdx_daily.fetch_trading_days(start_date, end_date)
|
||||
if days:
|
||||
_logger.info("交易日历 %s ~ %s 写入 %d 天", start_date, end_date, len(days))
|
||||
return days
|
||||
|
||||
sd = _format_date(start_date)
|
||||
ed = _format_date(end_date)
|
||||
|
||||
session = get_session()
|
||||
try:
|
||||
result = session.execute(
|
||||
select(TradingDay.date)
|
||||
.where(TradingDay.date >= sd)
|
||||
.where(TradingDay.date <= ed)
|
||||
.order_by(TradingDay.date)
|
||||
)
|
||||
days = [str(row[0]) for row in result]
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
if days and days[0] <= sd and days[-1] >= ed:
|
||||
_logger.info("交易日历 %s ~ %s 已有 %d 天，跳过", sd, ed, len(days))
|
||||
return
|
||||
|
||||
_logger.info("正在抓取交易日历 %s ~ %s...", start_date, end_date)
|
||||
days = _fetch_and_save(start_date, end_date)
|
||||
if not days:
|
||||
_logger.warning("无交易日数据")
|
||||
__all__ = ["fetch_trading_days", "get_trading_days"]
|
||||
|
||||
@@ -0,0 +1 @@
|
||||
"""Industry taxonomies, evidence parsing, and pure product classification."""
|
||||
@@ -0,0 +1,164 @@
|
||||
"""Ordered registry of industry taxonomies and their candidate boards."""
|
||||
|
||||
from ..models import ChainSpec, INDUSTRY_CHAIN_SOURCE
|
||||
from .technology import (
|
||||
SEMICONDUCTOR_DEFINITIONS,
|
||||
SEMICONDUCTOR_RULES,
|
||||
ROBOT_DEFINITIONS,
|
||||
ROBOT_RULES,
|
||||
SPACE_DEFINITIONS,
|
||||
SPACE_RULES,
|
||||
CPO_DEFINITIONS,
|
||||
CPO_RULES,
|
||||
LOW_ALTITUDE_DEFINITIONS,
|
||||
LOW_ALTITUDE_RULES,
|
||||
AIDC_DEFINITIONS,
|
||||
AIDC_RULES,
|
||||
MILITARY_ELECTRONICS_DEFINITIONS,
|
||||
MILITARY_ELECTRONICS_RULES,
|
||||
CONSUMER_ELECTRONICS_DEFINITIONS,
|
||||
CONSUMER_ELECTRONICS_RULES,
|
||||
SOFTWARE_CLOUD_DEFINITIONS,
|
||||
SOFTWARE_CLOUD_RULES,
|
||||
ELECTRONIC_COMPONENT_DEFINITIONS,
|
||||
ELECTRONIC_COMPONENT_RULES,
|
||||
COMMUNICATION_NETWORK_DEFINITIONS,
|
||||
COMMUNICATION_NETWORK_RULES,
|
||||
)
|
||||
from .energy_industry import (
|
||||
SOLID_STATE_BATTERY_DEFINITIONS,
|
||||
SOLID_STATE_BATTERY_RULES,
|
||||
COAL_CHEMICAL_DEFINITIONS,
|
||||
COAL_CHEMICAL_RULES,
|
||||
ENERGY_STORAGE_DEFINITIONS,
|
||||
ENERGY_STORAGE_RULES,
|
||||
PV_DEFINITIONS,
|
||||
PV_RULES,
|
||||
WIND_DEFINITIONS,
|
||||
WIND_RULES,
|
||||
HYDROGEN_DEFINITIONS,
|
||||
HYDROGEN_RULES,
|
||||
NEW_ENERGY_AUTO_DEFINITIONS,
|
||||
NEW_ENERGY_AUTO_RULES,
|
||||
PETROCHEMICAL_DEFINITIONS,
|
||||
PETROCHEMICAL_RULES,
|
||||
STEEL_DEFINITIONS,
|
||||
STEEL_RULES,
|
||||
NONFERROUS_DEFINITIONS,
|
||||
NONFERROUS_RULES,
|
||||
AUTO_AFTERMARKET_DEFINITIONS,
|
||||
AUTO_AFTERMARKET_RULES,
|
||||
)
|
||||
from .healthcare import (
|
||||
INNOVATIVE_DRUG_DEFINITIONS,
|
||||
INNOVATIVE_DRUG_RULES,
|
||||
MEDICAL_DEVICE_DEFINITIONS,
|
||||
MEDICAL_DEVICE_RULES,
|
||||
CHEMICAL_PHARMA_DEFINITIONS,
|
||||
CHEMICAL_PHARMA_RULES,
|
||||
TCM_DEFINITIONS,
|
||||
TCM_RULES,
|
||||
MEDICAL_SERVICE_DEFINITIONS,
|
||||
MEDICAL_SERVICE_RULES,
|
||||
PHARMA_DISTRIBUTION_DEFINITIONS,
|
||||
PHARMA_DISTRIBUTION_RULES,
|
||||
)
|
||||
from .consumer_agriculture import (
|
||||
FOOD_BEVERAGE_DEFINITIONS,
|
||||
FOOD_BEVERAGE_RULES,
|
||||
TOURISM_DEFINITIONS,
|
||||
TOURISM_RULES,
|
||||
RETAIL_DEFINITIONS,
|
||||
RETAIL_RULES,
|
||||
CROP_DEFINITIONS,
|
||||
CROP_RULES,
|
||||
LIVESTOCK_DEFINITIONS,
|
||||
LIVESTOCK_RULES,
|
||||
AQUACULTURE_DEFINITIONS,
|
||||
AQUACULTURE_RULES,
|
||||
PET_DEFINITIONS,
|
||||
PET_RULES,
|
||||
HOME_APPLIANCE_DEFINITIONS,
|
||||
HOME_APPLIANCE_RULES,
|
||||
TEXTILE_DEFINITIONS,
|
||||
TEXTILE_RULES,
|
||||
LIGHT_HOME_DEFINITIONS,
|
||||
LIGHT_HOME_RULES,
|
||||
)
|
||||
from .services_infrastructure import (
|
||||
INFRASTRUCTURE_DEFINITIONS,
|
||||
INFRASTRUCTURE_RULES,
|
||||
EDUCATION_DEFINITIONS,
|
||||
EDUCATION_RULES,
|
||||
HR_SERVICE_DEFINITIONS,
|
||||
HR_SERVICE_RULES,
|
||||
MEDIA_CONTENT_DEFINITIONS,
|
||||
MEDIA_CONTENT_RULES,
|
||||
TRANSPORT_LOGISTICS_DEFINITIONS,
|
||||
TRANSPORT_LOGISTICS_RULES,
|
||||
UTILITY_DEFINITIONS,
|
||||
UTILITY_RULES,
|
||||
ENVIRONMENT_DEFINITIONS,
|
||||
ENVIRONMENT_RULES,
|
||||
REAL_ESTATE_DEFINITIONS,
|
||||
REAL_ESTATE_RULES,
|
||||
BUILDING_MATERIAL_DEFINITIONS,
|
||||
BUILDING_MATERIAL_RULES,
|
||||
FINANCE_DEFINITIONS,
|
||||
FINANCE_RULES,
|
||||
BUSINESS_INDUSTRY_DEFINITIONS,
|
||||
BUSINESS_INDUSTRY_RULES,
|
||||
)
|
||||
|
||||
|
||||
CHAIN_SPECS = (
|
||||
ChainSpec("半导体", INDUSTRY_CHAIN_SOURCE, "thshy", "881121", SEMICONDUCTOR_DEFINITIONS, SEMICONDUCTOR_RULES, "CHAIN_SEMI_MID_OTHER"),
|
||||
ChainSpec("机器人", "industry_chain_robot", "gn", "300816", ROBOT_DEFINITIONS, ROBOT_RULES),
|
||||
ChainSpec("商业航天", "industry_chain_space", "gn", "309130", SPACE_DEFINITIONS, SPACE_RULES),
|
||||
ChainSpec("CPO", "industry_chain_cpo", "gn", "309049", CPO_DEFINITIONS, CPO_RULES),
|
||||
ChainSpec("固态电池", "industry_chain_ssb", "gn", "308294", SOLID_STATE_BATTERY_DEFINITIONS, SOLID_STATE_BATTERY_RULES),
|
||||
ChainSpec("煤化工", "industry_chain_coal", "gn", "300084", COAL_CHEMICAL_DEFINITIONS, COAL_CHEMICAL_RULES),
|
||||
ChainSpec("基础设施工程", "industry_chain_infra", "gn", "300841", INFRASTRUCTURE_DEFINITIONS, INFRASTRUCTURE_RULES),
|
||||
ChainSpec("低空经济", "industry_chain_la", "gn", "309115", LOW_ALTITUDE_DEFINITIONS, LOW_ALTITUDE_RULES),
|
||||
ChainSpec("AIDC", "industry_chain_aidc", "gn", "308642", AIDC_DEFINITIONS, AIDC_RULES),
|
||||
ChainSpec("储能", "industry_chain_es", "gn", "306380", ENERGY_STORAGE_DEFINITIONS, ENERGY_STORAGE_RULES),
|
||||
ChainSpec("光伏", "industry_chain_pv", "gn", "301079", PV_DEFINITIONS, PV_RULES),
|
||||
ChainSpec("风电", "industry_chain_wind", "gn", "300200", WIND_DEFINITIONS, WIND_RULES),
|
||||
ChainSpec("氢能源", "industry_chain_h2", "gn", "308491", HYDROGEN_DEFINITIONS, HYDROGEN_RULES),
|
||||
ChainSpec("创新药", "industry_chain_drug", "gn", "308014", INNOVATIVE_DRUG_DEFINITIONS, INNOVATIVE_DRUG_RULES),
|
||||
ChainSpec("医疗器械", "industry_chain_med", "gn", "301505", MEDICAL_DEVICE_DEFINITIONS, MEDICAL_DEVICE_RULES),
|
||||
ChainSpec("新能源汽车", "industry_chain_auto", "gn", "300008", NEW_ENERGY_AUTO_DEFINITIONS, NEW_ENERGY_AUTO_RULES, additional_boards=(("thshy", "881126"), ("thshy", "881125"))),
|
||||
ChainSpec("军工电子", "industry_chain_mil", "thshy", "881276", MILITARY_ELECTRONICS_DEFINITIONS, MILITARY_ELECTRONICS_RULES, additional_boards=(("gn", "309128"),)),
|
||||
ChainSpec("消费电子", "industry_chain_ce", "thshy", "881124", CONSUMER_ELECTRONICS_DEFINITIONS, CONSUMER_ELECTRONICS_RULES, additional_boards=(("gn", "308384"), ("gn", "309152"))),
|
||||
ChainSpec("软件云安全", "industry_chain_sw", "thshy", "881272", SOFTWARE_CLOUD_DEFINITIONS, SOFTWARE_CLOUD_RULES, additional_boards=(("gn", "309020"), ("gn", "300419"), ("gn", "300756"))),
|
||||
ChainSpec("石化精细化工", "industry_chain_chem", "thshy", "881107", PETROCHEMICAL_DEFINITIONS, PETROCHEMICAL_RULES, additional_boards=(("thshy", "881180"), ("thshy", "881108"), ("thshy", "881109"), ("thshy", "881264"), ("thshy", "881172"))),
|
||||
ChainSpec("化学制药", "industry_chain_pharm", "thshy", "881140", CHEMICAL_PHARMA_DEFINITIONS, CHEMICAL_PHARMA_RULES),
|
||||
ChainSpec("中药", "industry_chain_tcm", "thshy", "881141", TCM_DEFINITIONS, TCM_RULES),
|
||||
ChainSpec("医疗服务", "industry_chain_care", "thshy", "881175", MEDICAL_SERVICE_DEFINITIONS, MEDICAL_SERVICE_RULES),
|
||||
ChainSpec("医药流通", "industry_chain_dist", "thshy", "881143", PHARMA_DISTRIBUTION_DEFINITIONS, PHARMA_DISTRIBUTION_RULES),
|
||||
ChainSpec("食品饮料", "industry_chain_food", "thshy", "881134", FOOD_BEVERAGE_DEFINITIONS, FOOD_BEVERAGE_RULES, additional_boards=(("thshy", "881133"), ("thshy", "881273"))),
|
||||
ChainSpec("旅游酒店", "industry_chain_tour", "thshy", "881160", TOURISM_DEFINITIONS, TOURISM_RULES, additional_boards=(("gn", "301223"), ("gn", "308657"))),
|
||||
ChainSpec("零售商业", "industry_chain_shop", "thshy", "881158", RETAIL_DEFINITIONS, RETAIL_RULES, additional_boards=(("thshy", "881177"),)),
|
||||
ChainSpec("种植业", "industry_chain_crop", "thshy", "881101", CROP_DEFINITIONS, CROP_RULES, additional_boards=(("thshy", "881103"), ("gn", "308016"), ("gn", "300435"))),
|
||||
ChainSpec("畜禽养殖", "industry_chain_ani", "thshy", "881102", LIVESTOCK_DEFINITIONS, LIVESTOCK_RULES, additional_boards=(("thshy", "881103"), ("gn", "300451"), ("gn", "301576"), ("gn", "308537"))),
|
||||
ChainSpec("水产渔业", "industry_chain_aqua", "thshy", "881102", AQUACULTURE_DEFINITIONS, AQUACULTURE_RULES, additional_boards=(("thshy", "881103"),)),
|
||||
ChainSpec("宠物", "industry_chain_pet", "gn", "308588", PET_DEFINITIONS, PET_RULES, additional_boards=(("gn", "308537"),)),
|
||||
ChainSpec("家电", "industry_chain_home", "thshy", "881131", HOME_APPLIANCE_DEFINITIONS, HOME_APPLIANCE_RULES, additional_boards=(("thshy", "881132"), ("thshy", "881173"), ("gn", "300814"), ("gn", "300920"))),
|
||||
ChainSpec("纺织服装", "industry_chain_tex", "thshy", "881135", TEXTILE_DEFINITIONS, TEXTILE_RULES, additional_boards=(("thshy", "881136"),)),
|
||||
ChainSpec("轻工家居", "industry_chain_light", "thshy", "881139", LIGHT_HOME_DEFINITIONS, LIGHT_HOME_RULES, additional_boards=(("thshy", "881137"), ("thshy", "881138"))),
|
||||
ChainSpec("教育", "industry_chain_edu", "thshy", "881178", EDUCATION_DEFINITIONS, EDUCATION_RULES, additional_boards=(("gn", "301227"), ("gn", "301539"))),
|
||||
ChainSpec("人力资源服务", "industry_chain_hr", "thshy", "881179", HR_SERVICE_DEFINITIONS, HR_SERVICE_RULES),
|
||||
ChainSpec("传媒内容", "industry_chain_media", "thshy", "881164", MEDIA_CONTENT_DEFINITIONS, MEDIA_CONTENT_RULES, additional_boards=(("thshy", "881274"), ("thshy", "881275"))),
|
||||
ChainSpec("交通物流", "industry_chain_trans", "thshy", "881152", TRANSPORT_LOGISTICS_DEFINITIONS, TRANSPORT_LOGISTICS_RULES, additional_boards=(("thshy", "881148"), ("thshy", "881149"), ("thshy", "881151"))),
|
||||
ChainSpec("公用事业", "industry_chain_util", "thshy", "881145", UTILITY_DEFINITIONS, UTILITY_RULES, additional_boards=(("thshy", "881146"),)),
|
||||
ChainSpec("环保", "industry_chain_env", "thshy", "881181", ENVIRONMENT_DEFINITIONS, ENVIRONMENT_RULES, additional_boards=(("thshy", "881284"),)),
|
||||
ChainSpec("房地产", "industry_chain_re", "thshy", "881153", REAL_ESTATE_DEFINITIONS, REAL_ESTATE_RULES),
|
||||
ChainSpec("建筑材料", "industry_chain_mat", "thshy", "881115", BUILDING_MATERIAL_DEFINITIONS, BUILDING_MATERIAL_RULES),
|
||||
ChainSpec("钢铁", "industry_chain_steel", "thshy", "881112", STEEL_DEFINITIONS, STEEL_RULES),
|
||||
ChainSpec("有色金属", "industry_chain_metal", "thshy", "881168", NONFERROUS_DEFINITIONS, NONFERROUS_RULES, additional_boards=(("thshy", "881169"), ("thshy", "881170"), ("thshy", "881267"), ("thshy", "881114"), ("thshy", "881167"))),
|
||||
ChainSpec("金融服务", "industry_chain_fin", "thshy", "881155", FINANCE_DEFINITIONS, FINANCE_RULES, additional_boards=(("thshy", "881157"), ("thshy", "881156"), ("thshy", "881283"))),
|
||||
ChainSpec("电子元件", "industry_chain_ec", "thshy", "881122", ELECTRONIC_COMPONENT_DEFINITIONS, ELECTRONIC_COMPONENT_RULES, additional_boards=(("thshy", "881124"),)),
|
||||
ChainSpec("通信网络服务", "industry_chain_comm", "thshy", "881123", COMMUNICATION_NETWORK_DEFINITIONS, COMMUNICATION_NETWORK_RULES),
|
||||
ChainSpec("汽车后市场", "industry_chain_af", "thshy", "881125", AUTO_AFTERMARKET_DEFINITIONS, AUTO_AFTERMARKET_RULES, additional_boards=(("thshy", "881126"),)),
|
||||
ChainSpec("F10主营", "industry_chain_biz", "thshy", "881178", BUSINESS_INDUSTRY_DEFINITIONS, BUSINESS_INDUSTRY_RULES, "CHAIN_BIZ_MULTI"),
|
||||
)
|
||||
@@ -0,0 +1,388 @@
|
||||
"""Industry definitions and product rules: consumer agriculture."""
|
||||
|
||||
FOOD_BEVERAGE_DEFINITIONS = (
|
||||
("CHAIN_FOOD", "食品饮料产业链", "", 1),
|
||||
("CHAIN_FOOD_UP", "上游原料与供应", "CHAIN_FOOD", 2),
|
||||
("CHAIN_FOOD_UP_AGRI", "粮油糖与农产品原料", "CHAIN_FOOD_UP", 3),
|
||||
("CHAIN_FOOD_UP_DAIRY", "原奶与乳业牧场", "CHAIN_FOOD_UP", 3),
|
||||
("CHAIN_FOOD_UP_ADDITIVE", "食品配料与添加剂", "CHAIN_FOOD_UP", 3),
|
||||
("CHAIN_FOOD_MID", "中游食品制造", "CHAIN_FOOD", 2),
|
||||
("CHAIN_FOOD_MID_LIQUOR", "白酒", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_OTHER_ALCOHOL", "啤酒/黄酒/葡萄酒", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_DRINK", "软饮料与茶饮", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_DAIRY", "乳制品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_CONDIMENT", "调味品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_MEAT", "肉制品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_FROZEN", "速冻与预制食品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_BAKERY", "烘焙食品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_SNACK", "休闲零食", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_MID_NUTRITION", "保健与营养食品", "CHAIN_FOOD_MID", 3),
|
||||
("CHAIN_FOOD_DOWN", "下游渠道与餐饮供应", "CHAIN_FOOD", 2),
|
||||
("CHAIN_FOOD_DOWN_DISTRIBUTION", "食品经销与供应链", "CHAIN_FOOD_DOWN", 3),
|
||||
("CHAIN_FOOD_DOWN_RETAIL", "零食与食品零售", "CHAIN_FOOD_DOWN", 3),
|
||||
("CHAIN_FOOD_DOWN_CATERING", "餐饮供应链", "CHAIN_FOOD_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
FOOD_BEVERAGE_RULES = (
|
||||
("CHAIN_FOOD_UP_AGRI", ("食品用粮油原料", "食糖原料", "食品农产品原料", "面粉生产", "食用油原料")),
|
||||
("CHAIN_FOOD_UP_DAIRY", ("原奶生产", "奶牛养殖", "乳业牧场", "生鲜乳")),
|
||||
("CHAIN_FOOD_UP_ADDITIVE", ("食品配料", "食品添加剂", "食用香精香料", "食品酶制剂")),
|
||||
("CHAIN_FOOD_MID_LIQUOR", ("白酒生产", "白酒酿造", "浓香型白酒", "酱香型白酒", "清香型白酒")),
|
||||
("CHAIN_FOOD_MID_OTHER_ALCOHOL", ("啤酒生产", "黄酒生产", "葡萄酒生产", "果酒生产")),
|
||||
("CHAIN_FOOD_MID_DRINK", ("软饮料", "茶饮料", "包装饮用水", "植物蛋白饮料", "能量饮料", "果汁饮料")),
|
||||
("CHAIN_FOOD_MID_DAIRY", ("液态乳", "乳制品生产", "奶粉生产", "婴幼儿配方奶粉", "酸奶产品")),
|
||||
("CHAIN_FOOD_MID_CONDIMENT", ("酱油生产", "食醋生产", "复合调味料", "调味品生产", "火锅底料")),
|
||||
("CHAIN_FOOD_MID_MEAT", ("肉制品加工", "屠宰及肉类加工", "熟食肉制品")),
|
||||
("CHAIN_FOOD_MID_FROZEN", ("速冻食品", "预制菜", "预制食品", "冷冻调理食品")),
|
||||
("CHAIN_FOOD_MID_BAKERY", ("烘焙食品", "面包生产", "糕点生产", "冷冻烘焙")),
|
||||
("CHAIN_FOOD_MID_SNACK", ("休闲食品", "坚果炒货", "膨化食品", "卤制零食", "糖果巧克力")),
|
||||
("CHAIN_FOOD_MID_NUTRITION", ("保健食品", "营养食品", "特殊医学用途配方食品", "功能性食品", "体重管理食品")),
|
||||
("CHAIN_FOOD_DOWN_DISTRIBUTION", ("食品经销", "食品供应链服务", "酒类流通", "食品批发")),
|
||||
("CHAIN_FOOD_DOWN_RETAIL", ("零食连锁", "食品零售门店", "量贩零食")),
|
||||
("CHAIN_FOOD_DOWN_CATERING", ("餐饮供应链", "餐饮食材供应", "团餐供应链")),
|
||||
)
|
||||
|
||||
|
||||
TOURISM_DEFINITIONS = (
|
||||
("CHAIN_TOUR", "旅游酒店产业链", "", 1),
|
||||
("CHAIN_TOUR_UP", "上游资源与设施", "CHAIN_TOUR", 2),
|
||||
("CHAIN_TOUR_UP_RESOURCE", "文旅资源与目的地", "CHAIN_TOUR_UP", 3),
|
||||
("CHAIN_TOUR_UP_PROPERTY", "酒店物业与管理输出", "CHAIN_TOUR_UP", 3),
|
||||
("CHAIN_TOUR_UP_TRANSPORT", "旅游交通", "CHAIN_TOUR_UP", 3),
|
||||
("CHAIN_TOUR_MID", "中游运营", "CHAIN_TOUR", 2),
|
||||
("CHAIN_TOUR_MID_SCENIC", "景区运营", "CHAIN_TOUR_MID", 3),
|
||||
("CHAIN_TOUR_MID_THEME", "主题公园与演艺", "CHAIN_TOUR_MID", 3),
|
||||
("CHAIN_TOUR_MID_HOTEL", "酒店运营", "CHAIN_TOUR_MID", 3),
|
||||
("CHAIN_TOUR_MID_HOTEL_HIGH", "中高端酒店", "CHAIN_TOUR_MID_HOTEL", 4),
|
||||
("CHAIN_TOUR_MID_HOTEL_ECONOMY", "经济型酒店", "CHAIN_TOUR_MID_HOTEL", 4),
|
||||
("CHAIN_TOUR_MID_CATERING", "餐饮运营", "CHAIN_TOUR_MID", 3),
|
||||
("CHAIN_TOUR_MID_CRUISE", "邮轮与游船", "CHAIN_TOUR_MID", 3),
|
||||
("CHAIN_TOUR_DOWN", "下游渠道与服务", "CHAIN_TOUR", 2),
|
||||
("CHAIN_TOUR_DOWN_AGENCY", "旅行社与旅游服务", "CHAIN_TOUR_DOWN", 3),
|
||||
("CHAIN_TOUR_DOWN_OTA", "在线旅游平台", "CHAIN_TOUR_DOWN", 3),
|
||||
("CHAIN_TOUR_DOWN_DUTYFREE", "旅游零售与免税", "CHAIN_TOUR_DOWN", 3),
|
||||
("CHAIN_TOUR_DOWN_MARKETING", "文旅营销与会展", "CHAIN_TOUR_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
TOURISM_RULES = (
|
||||
("CHAIN_TOUR_UP_RESOURCE", ("旅游目的地开发", "文旅资源开发", "旅游资源开发", "文旅项目开发")),
|
||||
("CHAIN_TOUR_UP_PROPERTY", ("酒店管理输出", "酒店品牌管理", "酒店资产管理", "酒店物业运营")),
|
||||
("CHAIN_TOUR_UP_TRANSPORT", ("旅游客运", "旅游交通", "景区索道", "旅游专线", "旅游包车")),
|
||||
("CHAIN_TOUR_MID_SCENIC", ("景区运营", "旅游景区经营", "自然景区", "文化景区")),
|
||||
("CHAIN_TOUR_MID_THEME", ("主题公园运营", "旅游演艺", "文化演艺项目", "室内主题乐园")),
|
||||
("CHAIN_TOUR_MID_HOTEL_HIGH", ("中高端酒店", "高端酒店运营", "度假酒店运营")),
|
||||
("CHAIN_TOUR_MID_HOTEL_ECONOMY", ("经济型酒店", "有限服务酒店", "快捷酒店")),
|
||||
("CHAIN_TOUR_MID_HOTEL", ("酒店运营", "连锁酒店", "酒店住宿业务")),
|
||||
("CHAIN_TOUR_MID_CATERING", ("连锁餐饮", "餐饮门店运营", "餐饮服务业务")),
|
||||
("CHAIN_TOUR_MID_CRUISE", ("邮轮旅游运营", "旅游游船运营", "豪华游轮运营业务", "水上旅游客运")),
|
||||
("CHAIN_TOUR_DOWN_AGENCY", ("旅行社业务", "旅游产品服务", "出境游服务", "入境游服务")),
|
||||
("CHAIN_TOUR_DOWN_OTA", ("在线旅游平台", "在线旅游服务", "ota平台", "酒店在线预订")),
|
||||
("CHAIN_TOUR_DOWN_DUTYFREE", ("以免税为主的旅游零售业务", "旅游零售运营商", "免税商品销售", "免税店运营", "旅游零售业务")),
|
||||
("CHAIN_TOUR_DOWN_MARKETING", ("旅游营销服务", "文旅营销", "会展旅游", "会议会展服务")),
|
||||
)
|
||||
|
||||
|
||||
RETAIL_DEFINITIONS = (
|
||||
("CHAIN_SHOP", "零售商业产业链", "", 1),
|
||||
("CHAIN_SHOP_UP", "上游供应链与技术", "CHAIN_SHOP", 2),
|
||||
("CHAIN_SHOP_UP_LOGISTICS", "零售仓配与供应链", "CHAIN_SHOP_UP", 3),
|
||||
("CHAIN_SHOP_UP_DIGITAL", "零售数字化", "CHAIN_SHOP_UP", 3),
|
||||
("CHAIN_SHOP_MID", "中游零售业态", "CHAIN_SHOP", 2),
|
||||
("CHAIN_SHOP_MID_DEPARTMENT", "百货与购物中心", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_MID_SUPERMARKET", "商超与便利店", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_MID_JEWELRY", "黄金珠宝零售", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_MID_APPLIANCE", "家电与3C连锁", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_MID_PHARMA", "药店零售", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_MID_SPECIALTY", "专业连锁零售", "CHAIN_SHOP_MID", 3),
|
||||
("CHAIN_SHOP_DOWN", "下游电商与新零售", "CHAIN_SHOP", 2),
|
||||
("CHAIN_SHOP_DOWN_PLATFORM", "综合电商平台", "CHAIN_SHOP_DOWN", 3),
|
||||
("CHAIN_SHOP_DOWN_CROSS", "跨境电商", "CHAIN_SHOP_DOWN", 3),
|
||||
("CHAIN_SHOP_DOWN_LIVE", "直播与内容电商", "CHAIN_SHOP_DOWN", 3),
|
||||
("CHAIN_SHOP_DOWN_INSTANT", "即时零售", "CHAIN_SHOP_DOWN", 3),
|
||||
("CHAIN_SHOP_DOWN_BRAND", "品牌零售与出海", "CHAIN_SHOP_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
RETAIL_RULES = (
|
||||
("CHAIN_SHOP_UP_LOGISTICS", ("零售仓储配送", "零售供应链服务", "商品供应链管理", "零售物流中心")),
|
||||
("CHAIN_SHOP_UP_DIGITAL", ("零售数字化系统", "智慧零售解决方案", "门店数字化", "零售saas")),
|
||||
("CHAIN_SHOP_MID_DEPARTMENT", ("百货零售", "购物中心运营", "百货商场经营", "商业综合体运营")),
|
||||
("CHAIN_SHOP_MID_SUPERMARKET", ("连锁超市", "便利店运营", "商超零售", "生鲜超市")),
|
||||
("CHAIN_SHOP_MID_JEWELRY", ("黄金珠宝零售", "珠宝连锁门店", "黄金饰品销售")),
|
||||
("CHAIN_SHOP_MID_APPLIANCE", ("家电连锁零售", "3c产品零售", "消费电子零售门店")),
|
||||
("CHAIN_SHOP_MID_PHARMA", ("连锁药店", "医药零售门店", "零售药房")),
|
||||
("CHAIN_SHOP_MID_SPECIALTY", ("专业连锁零售", "家居连锁零售", "眼镜连锁零售", "母婴连锁零售")),
|
||||
("CHAIN_SHOP_DOWN_PLATFORM", ("综合电商平台", "电子商务平台运营", "互联网零售平台")),
|
||||
("CHAIN_SHOP_DOWN_CROSS", ("跨境电商平台", "跨境电商业务", "海外电商销售")),
|
||||
("CHAIN_SHOP_DOWN_LIVE", ("直播电商", "内容电商", "短视频电商", "电商代运营")),
|
||||
("CHAIN_SHOP_DOWN_INSTANT", ("即时零售平台", "到家零售", "小时达零售")),
|
||||
("CHAIN_SHOP_DOWN_BRAND", ("品牌零售出海", "自有品牌零售", "品牌电商销售")),
|
||||
)
|
||||
|
||||
|
||||
CROP_DEFINITIONS = (
|
||||
("CHAIN_CROP", "种植业产业链", "", 1),
|
||||
("CHAIN_CROP_UP", "上游种源与农资", "CHAIN_CROP", 2),
|
||||
("CHAIN_CROP_UP_SEED", "种子研发与繁育", "CHAIN_CROP_UP", 3),
|
||||
("CHAIN_CROP_UP_SEED_CORN", "玉米种子", "CHAIN_CROP_UP_SEED", 4),
|
||||
("CHAIN_CROP_UP_SEED_RICE", "水稻种子", "CHAIN_CROP_UP_SEED", 4),
|
||||
("CHAIN_CROP_UP_SEED_OTHER", "小麦/蔬菜等种子", "CHAIN_CROP_UP_SEED", 4),
|
||||
("CHAIN_CROP_UP_BIOTECH", "生物育种", "CHAIN_CROP_UP", 3),
|
||||
("CHAIN_CROP_UP_SERVICE", "农业技术与植保服务", "CHAIN_CROP_UP", 3),
|
||||
("CHAIN_CROP_MID", "中游种植", "CHAIN_CROP", 2),
|
||||
("CHAIN_CROP_MID_GRAIN", "粮食作物", "CHAIN_CROP_MID", 3),
|
||||
("CHAIN_CROP_MID_OIL", "油料与糖料作物", "CHAIN_CROP_MID", 3),
|
||||
("CHAIN_CROP_MID_FRUIT", "果蔬与园艺", "CHAIN_CROP_MID", 3),
|
||||
("CHAIN_CROP_MID_COTTON", "棉花与经济作物", "CHAIN_CROP_MID", 3),
|
||||
("CHAIN_CROP_MID_FORESTRY", "林业与林产品", "CHAIN_CROP_MID", 3),
|
||||
("CHAIN_CROP_DOWN", "下游加工与流通", "CHAIN_CROP", 2),
|
||||
("CHAIN_CROP_DOWN_PROCESS", "农产品初加工", "CHAIN_CROP_DOWN", 3),
|
||||
("CHAIN_CROP_DOWN_STORAGE", "粮食仓储与贸易", "CHAIN_CROP_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
CROP_RULES = (
|
||||
("CHAIN_CROP_UP_SEED_CORN", ("玉米种子", "玉米育种", "玉米种业")),
|
||||
("CHAIN_CROP_UP_SEED_RICE", ("水稻种子", "杂交水稻种子", "水稻育种")),
|
||||
("CHAIN_CROP_UP_SEED_OTHER", ("小麦种子", "蔬菜种子", "瓜果种子", "花卉种子")),
|
||||
("CHAIN_CROP_UP_SEED", ("农作物种子", "种子繁育", "种业研发")),
|
||||
("CHAIN_CROP_UP_BIOTECH", ("生物育种", "转基因育种", "基因编辑育种", "分子育种")),
|
||||
("CHAIN_CROP_UP_SERVICE", ("农业植保服务", "种植技术服务", "农业社会化服务", "农技服务")),
|
||||
("CHAIN_CROP_MID_GRAIN", ("水稻种植", "玉米种植", "小麦种植", "粮食作物种植")),
|
||||
("CHAIN_CROP_MID_OIL", ("大豆种植", "油料作物种植", "甘蔗种植", "甜菜种植")),
|
||||
("CHAIN_CROP_MID_FRUIT", ("水果种植", "蔬菜种植", "果蔬种植", "园艺作物")),
|
||||
("CHAIN_CROP_MID_COTTON", ("棉花种植", "橡胶种植", "茶叶种植", "经济作物种植")),
|
||||
("CHAIN_CROP_MID_FORESTRY", ("林木种植", "林业经营", "木材采运", "林产品")),
|
||||
("CHAIN_CROP_DOWN_PROCESS", ("粮食初加工", "农产品初加工", "棉花加工", "果蔬初加工")),
|
||||
("CHAIN_CROP_DOWN_STORAGE", ("粮食仓储", "粮食贸易", "农产品仓储", "粮食收购")),
|
||||
)
|
||||
|
||||
|
||||
LIVESTOCK_DEFINITIONS = (
|
||||
("CHAIN_ANI", "畜禽养殖产业链", "", 1),
|
||||
("CHAIN_ANI_UP", "上游育种饲料与动保", "CHAIN_ANI", 2),
|
||||
("CHAIN_ANI_UP_BREED", "畜禽育种", "CHAIN_ANI_UP", 3),
|
||||
("CHAIN_ANI_UP_FEED", "饲料与饲料添加剂", "CHAIN_ANI_UP", 3),
|
||||
("CHAIN_ANI_UP_VACCINE", "动物疫苗", "CHAIN_ANI_UP", 3),
|
||||
("CHAIN_ANI_UP_DRUG", "兽药与动保服务", "CHAIN_ANI_UP", 3),
|
||||
("CHAIN_ANI_MID", "中游养殖", "CHAIN_ANI", 2),
|
||||
("CHAIN_ANI_MID_PIG", "生猪养殖", "CHAIN_ANI_MID", 3),
|
||||
("CHAIN_ANI_MID_POULTRY", "禽类养殖", "CHAIN_ANI_MID", 3),
|
||||
("CHAIN_ANI_MID_CATTLE", "肉牛与羊养殖", "CHAIN_ANI_MID", 3),
|
||||
("CHAIN_ANI_MID_DAIRY", "奶牛养殖", "CHAIN_ANI_MID", 3),
|
||||
("CHAIN_ANI_DOWN", "下游屠宰加工与流通", "CHAIN_ANI", 2),
|
||||
("CHAIN_ANI_DOWN_SLAUGHTER", "生猪屠宰与猪肉", "CHAIN_ANI_DOWN", 3),
|
||||
("CHAIN_ANI_DOWN_POULTRY", "禽肉与禽蛋加工", "CHAIN_ANI_DOWN", 3),
|
||||
("CHAIN_ANI_DOWN_BEEF", "牛羊肉加工", "CHAIN_ANI_DOWN", 3),
|
||||
("CHAIN_ANI_DOWN_COLD", "肉类冷链与贸易", "CHAIN_ANI_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
LIVESTOCK_RULES = (
|
||||
("CHAIN_ANI_UP_BREED", ("畜禽育种", "种猪繁育", "白羽肉鸡育种", "祖代种鸡", "种牛繁育")),
|
||||
("CHAIN_ANI_UP_FEED", ("猪饲料", "禽饲料", "反刍饲料", "畜禽饲料", "饲料添加剂")),
|
||||
("CHAIN_ANI_UP_VACCINE", ("猪用疫苗", "禽用疫苗", "动物疫苗", "兽用生物制品")),
|
||||
("CHAIN_ANI_UP_DRUG", ("猪用兽药", "禽用兽药", "动物保健产品", "兽药制剂", "养殖技术服务")),
|
||||
("CHAIN_ANI_MID_PIG", ("生猪养殖", "商品猪养殖", "生猪出栏")),
|
||||
("CHAIN_ANI_MID_POULTRY", ("肉鸡养殖", "蛋鸡养殖", "白羽鸡养殖", "黄羽鸡养殖", "禽类养殖")),
|
||||
("CHAIN_ANI_MID_CATTLE", ("肉牛养殖", "肉羊养殖", "牛羊养殖")),
|
||||
("CHAIN_ANI_MID_DAIRY", ("奶牛养殖", "乳牛养殖", "原奶生产")),
|
||||
("CHAIN_ANI_DOWN_SLAUGHTER", ("生猪屠宰", "猪肉产品", "猪肉分割")),
|
||||
("CHAIN_ANI_DOWN_POULTRY", ("禽类屠宰", "鸡肉产品", "鸭肉产品", "禽蛋加工")),
|
||||
("CHAIN_ANI_DOWN_BEEF", ("牛羊屠宰", "牛肉加工", "羊肉加工")),
|
||||
("CHAIN_ANI_DOWN_COLD", ("肉类冷链", "肉类贸易", "冻肉进口")),
|
||||
)
|
||||
|
||||
|
||||
AQUACULTURE_DEFINITIONS = (
|
||||
("CHAIN_AQUA", "水产渔业产业链", "", 1),
|
||||
("CHAIN_AQUA_UP", "上游苗种饲料与动保", "CHAIN_AQUA", 2),
|
||||
("CHAIN_AQUA_UP_SEED", "水产苗种", "CHAIN_AQUA_UP", 3),
|
||||
("CHAIN_AQUA_UP_FEED", "水产饲料", "CHAIN_AQUA_UP", 3),
|
||||
("CHAIN_AQUA_UP_HEALTH", "水产动保", "CHAIN_AQUA_UP", 3),
|
||||
("CHAIN_AQUA_MID", "中游养殖与捕捞", "CHAIN_AQUA", 2),
|
||||
("CHAIN_AQUA_MID_MARINE", "海水养殖", "CHAIN_AQUA_MID", 3),
|
||||
("CHAIN_AQUA_MID_FRESH", "淡水养殖", "CHAIN_AQUA_MID", 3),
|
||||
("CHAIN_AQUA_MID_SPECIAL", "特种水产养殖", "CHAIN_AQUA_MID", 3),
|
||||
("CHAIN_AQUA_MID_FISHING", "海洋捕捞", "CHAIN_AQUA_MID", 3),
|
||||
("CHAIN_AQUA_DOWN", "下游加工与流通", "CHAIN_AQUA", 2),
|
||||
("CHAIN_AQUA_DOWN_PROCESS", "水产品加工", "CHAIN_AQUA_DOWN", 3),
|
||||
("CHAIN_AQUA_DOWN_COLD", "水产冷链", "CHAIN_AQUA_DOWN", 3),
|
||||
("CHAIN_AQUA_DOWN_TRADE", "水产品贸易与品牌", "CHAIN_AQUA_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
AQUACULTURE_RULES = (
|
||||
("CHAIN_AQUA_UP_SEED", ("水产苗种", "鱼苗繁育", "虾苗繁育", "海参苗种")),
|
||||
("CHAIN_AQUA_UP_FEED", ("水产饲料", "鱼饲料", "虾饲料", "特种水产饲料")),
|
||||
("CHAIN_AQUA_UP_HEALTH", ("水产动保", "水产用兽药", "水产疫苗", "水产养殖技术服务")),
|
||||
("CHAIN_AQUA_MID_MARINE", ("海水养殖", "海参养殖", "海水鱼养殖", "贝类养殖")),
|
||||
("CHAIN_AQUA_MID_FRESH", ("淡水养殖", "淡水鱼养殖", "池塘养殖")),
|
||||
("CHAIN_AQUA_MID_SPECIAL", ("对虾养殖", "鳗鱼养殖", "小龙虾养殖", "特种水产养殖")),
|
||||
("CHAIN_AQUA_MID_FISHING", ("海洋捕捞", "远洋捕捞", "渔业捕捞")),
|
||||
("CHAIN_AQUA_DOWN_PROCESS", ("水产品加工", "海产品加工", "鱼类加工", "虾类加工")),
|
||||
("CHAIN_AQUA_DOWN_COLD", ("水产冷链", "海产品冷链", "冷冻水产品")),
|
||||
("CHAIN_AQUA_DOWN_TRADE", ("水产品贸易", "海产品销售", "水产品品牌")),
|
||||
)
|
||||
|
||||
|
||||
PET_DEFINITIONS = (
|
||||
("CHAIN_PET", "宠物产业链", "", 1),
|
||||
("CHAIN_PET_UP", "上游原料与健康", "CHAIN_PET", 2),
|
||||
("CHAIN_PET_UP_RAW", "宠物食品原料", "CHAIN_PET_UP", 3),
|
||||
("CHAIN_PET_UP_VACCINE", "宠物疫苗", "CHAIN_PET_UP", 3),
|
||||
("CHAIN_PET_UP_DRUG", "宠物药品", "CHAIN_PET_UP", 3),
|
||||
("CHAIN_PET_MID", "中游产品制造", "CHAIN_PET", 2),
|
||||
("CHAIN_PET_MID_FOOD", "宠物主粮与综合食品", "CHAIN_PET_MID", 3),
|
||||
("CHAIN_PET_MID_SNACK", "宠物零食", "CHAIN_PET_MID", 3),
|
||||
("CHAIN_PET_MID_SUPPLY", "宠物用品", "CHAIN_PET_MID", 3),
|
||||
("CHAIN_PET_DOWN", "下游渠道与服务", "CHAIN_PET", 2),
|
||||
("CHAIN_PET_DOWN_HOSPITAL", "宠物医院", "CHAIN_PET_DOWN", 3),
|
||||
("CHAIN_PET_DOWN_RETAIL", "宠物零售与电商", "CHAIN_PET_DOWN", 3),
|
||||
("CHAIN_PET_DOWN_SERVICE", "宠物服务", "CHAIN_PET_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
PET_RULES = (
|
||||
("CHAIN_PET_UP_RAW", ("宠物食品原料", "宠物粮原料", "宠物食品肉粉")),
|
||||
("CHAIN_PET_UP_VACCINE", ("宠物疫苗", "犬用疫苗", "猫用疫苗")),
|
||||
("CHAIN_PET_UP_DRUG", ("宠物药品", "宠物用兽药", "犬猫药品", "宠物驱虫药")),
|
||||
("CHAIN_PET_MID_FOOD", ("宠物食品的研发", "宠物食品研发", "宠物食品生产", "宠物食品制造", "宠物主粮", "犬粮", "猫粮", "宠物干粮", "宠物湿粮")),
|
||||
("CHAIN_PET_MID_SNACK", ("宠物零食", "狗零食", "猫零食", "宠物咬胶")),
|
||||
("CHAIN_PET_MID_SUPPLY", ("宠物牵引用具", "宠物注塑玩具", "宠物训练器", "止吠器", "猫砂生产", "宠物玩具生产", "宠物清洁用品")),
|
||||
("CHAIN_PET_DOWN_HOSPITAL", ("宠物医院运营", "连锁宠物医院", "宠物诊疗服务", "宠物医疗服务")),
|
||||
("CHAIN_PET_DOWN_RETAIL", ("宠物电商", "宠物用品零售", "宠物连锁门店")),
|
||||
("CHAIN_PET_DOWN_SERVICE", ("宠物美容", "宠物寄养", "宠物保险", "宠物服务平台")),
|
||||
)
|
||||
|
||||
|
||||
HOME_APPLIANCE_DEFINITIONS = (
|
||||
("CHAIN_HOME", "家电产业链", "", 1),
|
||||
("CHAIN_HOME_UP", "上游核心部件", "CHAIN_HOME", 2),
|
||||
("CHAIN_HOME_UP_COMPRESSOR", "压缩机与电机", "CHAIN_HOME_UP", 3),
|
||||
("CHAIN_HOME_UP_CONTROL", "控制器与功率器件", "CHAIN_HOME_UP", 3),
|
||||
("CHAIN_HOME_UP_THERMAL", "换热器与制冷部件", "CHAIN_HOME_UP", 3),
|
||||
("CHAIN_HOME_UP_DISPLAY", "面板与显示模组", "CHAIN_HOME_UP", 3),
|
||||
("CHAIN_HOME_MID", "中游整机制造", "CHAIN_HOME", 2),
|
||||
("CHAIN_HOME_MID_INTEGRATED", "综合家电", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_AC", "空调", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_FRIDGE", "冰箱冷柜", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_WASHER", "洗衣机", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_KITCHEN", "厨卫电器", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_SMALL", "小家电", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_TV", "电视与智能显示", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_CLEAN", "清洁电器", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_MID_SMART", "智能家居系统", "CHAIN_HOME_MID", 3),
|
||||
("CHAIN_HOME_DOWN", "下游渠道与服务", "CHAIN_HOME", 2),
|
||||
("CHAIN_HOME_DOWN_ODM", "家电ODM/OEM", "CHAIN_HOME_DOWN", 3),
|
||||
("CHAIN_HOME_DOWN_CHANNEL", "家电渠道与零售", "CHAIN_HOME_DOWN", 3),
|
||||
("CHAIN_HOME_DOWN_SERVICE", "安装与售后服务", "CHAIN_HOME_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
HOME_APPLIANCE_RULES = (
|
||||
("CHAIN_HOME_UP_COMPRESSOR", ("家电压缩机", "空调压缩机", "冰箱压缩机", "家电电机")),
|
||||
("CHAIN_HOME_UP_CONTROL", ("家电控制器", "家用电器控制器", "生活电器控制器", "家电变频控制", "家电功率模块", "家电智能控制器")),
|
||||
("CHAIN_HOME_UP_THERMAL", ("家电换热器", "制冷配件", "空调阀件", "家电热交换器")),
|
||||
("CHAIN_HOME_UP_DISPLAY", ("电视面板", "家电显示模组", "智能电视显示屏")),
|
||||
("CHAIN_HOME_MID_INTEGRATED", ("家电的研发、生产及销售", "家用电器的研发、生产和销售", "电视机、冰箱、空调", "电视、空调、冰箱", "电冰箱、冰柜及空调、洗衣机", "彩电业务、白电业务", "多品类家电")),
|
||||
("CHAIN_HOME_MID_AC", ("家用空调整机", "中央空调整机", "移动空调整机", "空调整机研发")),
|
||||
("CHAIN_HOME_MID_FRIDGE", ("冰箱整机", "冷柜整机", "冰箱冷柜产品", "家用冰箱")),
|
||||
("CHAIN_HOME_MID_WASHER", ("洗衣机整机", "洗衣机产品", "家用洗衣机", "干衣机整机")),
|
||||
("CHAIN_HOME_MID_KITCHEN", ("厨房电器产品", "厨卫电器产品", "厨房电器的研发", "厨卫电器的研发", "吸油烟机生产", "燃气灶生产", "热水器整机", "净水器整机")),
|
||||
("CHAIN_HOME_MID_CLEAN", ("智能清洁电器的研发", "清洁健康家电", "清洁电器产品", "清洁电器整机", "家用、商用服务机器人", "科沃斯品牌服务机器人", "智能生活电器", "扫地机器人整机", "扫地机器人和洗地机研发生产", "扫地机器人研发生产", "扫地机器人制造", "洗地机整机", "洗地机研发生产", "吸尘器整机")),
|
||||
("CHAIN_HOME_MID_SMALL", ("小家电系列产品", "小家电研发、设计、生产和销售", "小型家电", "西式小家电", "厨房小家电", "生活小家电", "个护小家电", "小家电产品", "小家电整机", "小家电", "小家电的研发", "小家电的生产", "生活电器产品", "生活电器的研发", "个人护理电器整机")),
|
||||
("CHAIN_HOME_MID_TV", ("智慧显示终端", "彩电业务", "智能电视整机", "电视整机", "电视产品", "智能显示终端", "激光电视整机", "液晶电视的研发", "液晶电视生产")),
|
||||
("CHAIN_HOME_MID_SMART", ("智能家居业务", "智能家居系统", "智能家居解决方案", "全屋智能业务", "智能家居平台运营")),
|
||||
("CHAIN_HOME_DOWN_ODM", ("家电odm", "家电oem", "家电代工", "家电整机代工", "清洁电器整机代工", "吸尘器整机代工")),
|
||||
("CHAIN_HOME_DOWN_CHANNEL", ("家电零售", "家电连锁", "家电电商销售")),
|
||||
("CHAIN_HOME_DOWN_SERVICE", ("家电安装服务", "家电售后服务", "家电维修服务")),
|
||||
)
|
||||
|
||||
|
||||
TEXTILE_DEFINITIONS = (
|
||||
("CHAIN_TEX", "纺织服装产业链", "", 1),
|
||||
("CHAIN_TEX_UP", "上游纤维纱线与辅料", "CHAIN_TEX", 2),
|
||||
("CHAIN_TEX_UP_FIBER", "天然与化学纤维", "CHAIN_TEX_UP", 3),
|
||||
("CHAIN_TEX_UP_YARN", "纱线", "CHAIN_TEX_UP", 3),
|
||||
("CHAIN_TEX_UP_ACCESSORY", "服装辅料", "CHAIN_TEX_UP", 3),
|
||||
("CHAIN_TEX_UP_EQUIPMENT", "纺织服装设备", "CHAIN_TEX_UP", 3),
|
||||
("CHAIN_TEX_MID", "中游纺织与制造", "CHAIN_TEX", 2),
|
||||
("CHAIN_TEX_MID_FABRIC", "面料织造", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_MID_NONWOVEN", "非织造材料", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_MID_DYE", "印染与后整理", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_MID_GARMENT", "成衣制造与代工", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_MID_HOME", "家纺制造", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_MID_SHOE", "鞋履制造", "CHAIN_TEX_MID", 3),
|
||||
("CHAIN_TEX_DOWN", "下游品牌与渠道", "CHAIN_TEX", 2),
|
||||
("CHAIN_TEX_DOWN_SPORT", "运动户外品牌", "CHAIN_TEX_DOWN", 3),
|
||||
("CHAIN_TEX_DOWN_CASUAL", "休闲与大众服饰", "CHAIN_TEX_DOWN", 3),
|
||||
("CHAIN_TEX_DOWN_HIGH", "中高端服饰", "CHAIN_TEX_DOWN", 3),
|
||||
("CHAIN_TEX_DOWN_UNIFORM", "职业装与校服", "CHAIN_TEX_DOWN", 3),
|
||||
("CHAIN_TEX_DOWN_HOME", "家纺品牌", "CHAIN_TEX_DOWN", 3),
|
||||
("CHAIN_TEX_DOWN_EXPORT", "服装出口与跨境渠道", "CHAIN_TEX_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
TEXTILE_RULES = (
|
||||
("CHAIN_TEX_UP_FIBER", ("棉纤维", "羊毛纤维", "涤纶纤维", "锦纶纤维", "再生纤维素纤维", "化学纤维", "羽绒产品")),
|
||||
("CHAIN_TEX_UP_YARN", ("纱线", "棉纱", "色纱", "色纺纱", "功能纱线", "工业纱线", "羊毛毛条")),
|
||||
("CHAIN_TEX_UP_ACCESSORY", ("服装辅料", "服饰辅料", "拉链", "钮扣", "纽扣", "衬布生产")),
|
||||
("CHAIN_TEX_UP_EQUIPMENT", ("纺织机械", "纺机设备", "缝制机械", "工业缝纫机", "服装智能制造设备", "针织机械")),
|
||||
("CHAIN_TEX_MID_DYE", ("纺织印染业务", "印染业务", "印染加工为主业", "纺织印染加工", "印染加工", "面料染整加工", "染整受托加工", "纺织后整理加工")),
|
||||
("CHAIN_TEX_MID_NONWOVEN", ("非织造材料", "非织造布", "无纺布")),
|
||||
("CHAIN_TEX_MID_FABRIC", ("纺织面料", "针织面料", "梭织面料", "功能性面料", "面料", "色织布", "坯布", "产业用布", "革基布", "装饰面料")),
|
||||
("CHAIN_TEX_DOWN_SPORT", ("运动服饰品牌", "户外服装品牌", "运动鞋服品牌", "户外用品", "户外服饰", "运动鞋")),
|
||||
("CHAIN_TEX_DOWN_UNIFORM", ("职业装", "校服")),
|
||||
("CHAIN_TEX_DOWN_CASUAL", ("休闲服饰品牌", "大众服装品牌", "童装品牌", "休闲服饰", "儿童服饰", "男装", "童装", "内衣", "贴身服饰")),
|
||||
("CHAIN_TEX_DOWN_HIGH", ("高端服装品牌", "中高端女装", "中高端男装", "奢侈服饰品牌", "中高端服饰", "中高档服装", "品牌时装", "品牌服装")),
|
||||
("CHAIN_TEX_MID_GARMENT", ("服装代工", "成衣制造", "成衣", "服装生产", "服装制造", "服装智能制造", "制衣业务", "制衣生产", "服装oem", "服装odm")),
|
||||
("CHAIN_TEX_MID_HOME", ("家纺业务", "家用纺织品制造", "家用纺织品", "床上用品生产", "床上用品", "家纺产品制造", "家纺产品", "毛毯", "巾被系列产品")),
|
||||
("CHAIN_TEX_MID_SHOE", ("鞋履制造", "鞋履生产", "鞋履", "皮鞋", "运动鞋代工", "鞋材生产")),
|
||||
("CHAIN_TEX_DOWN_HOME", ("家纺品牌", "床上用品品牌")),
|
||||
("CHAIN_TEX_DOWN_EXPORT", ("服装出口", "纺织品出口", "服饰跨境电商")),
|
||||
)
|
||||
|
||||
|
||||
LIGHT_HOME_DEFINITIONS = (
|
||||
("CHAIN_LIGHT", "轻工家居产业链", "", 1),
|
||||
("CHAIN_LIGHT_UP", "上游材料", "CHAIN_LIGHT", 2),
|
||||
("CHAIN_LIGHT_UP_PULP", "木浆与废纸", "CHAIN_LIGHT_UP", 3),
|
||||
("CHAIN_LIGHT_UP_WOOD", "木材与板材", "CHAIN_LIGHT_UP", 3),
|
||||
("CHAIN_LIGHT_UP_METAL", "包装金属与塑料", "CHAIN_LIGHT_UP", 3),
|
||||
("CHAIN_LIGHT_UP_EQUIPMENT", "轻工制造设备", "CHAIN_LIGHT_UP", 3),
|
||||
("CHAIN_LIGHT_MID", "中游制造", "CHAIN_LIGHT", 2),
|
||||
("CHAIN_LIGHT_MID_PAPER_CULTURE", "文化纸", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_PAPER_PACKAGE", "包装纸", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_PAPER_TISSUE", "生活用纸", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_PACKAGE_PAPER", "纸包装", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_PACKAGE_METAL", "金属与塑料包装", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_FURNITURE_CUSTOM", "定制家居", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_FURNITURE_SOFT", "软体与成品家具", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_HARDWARE", "家居五金与卫浴", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_MID_STATIONERY", "文具与文娱用品", "CHAIN_LIGHT_MID", 3),
|
||||
("CHAIN_LIGHT_DOWN", "下游渠道与服务", "CHAIN_LIGHT", 2),
|
||||
("CHAIN_LIGHT_DOWN_RETAIL", "家居零售与整装", "CHAIN_LIGHT_DOWN", 3),
|
||||
("CHAIN_LIGHT_DOWN_EXPORT", "家居用品出口", "CHAIN_LIGHT_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
LIGHT_HOME_RULES = (
|
||||
("CHAIN_LIGHT_UP_PULP", ("木浆生产", "纸浆生产", "废纸回收", "再生纸浆")),
|
||||
("CHAIN_LIGHT_UP_WOOD", ("家具板材", "人造板生产", "木材加工", "刨花板")),
|
||||
("CHAIN_LIGHT_UP_METAL", ("包装铝材", "包装用马口铁", "包装塑料薄膜")),
|
||||
("CHAIN_LIGHT_UP_EQUIPMENT", ("瓦楞纸箱设备", "纸箱印刷机械", "造纸机械", "造纸生产线", "生活用纸智能装备", "生活用纸生产线", "人造板生产线", "板材生产装备", "包装机械设备", "家具制造设备")),
|
||||
("CHAIN_LIGHT_MID_PAPER_CULTURE", ("文化纸", "双胶纸", "铜版纸", "新闻纸")),
|
||||
("CHAIN_LIGHT_MID_PAPER_PACKAGE", ("箱板纸", "瓦楞原纸", "白卡纸", "包装纸生产")),
|
||||
("CHAIN_LIGHT_MID_PAPER_TISSUE", ("生活用纸生产", "生活用纸产品", "生活用纸", "卫生纸生产", "纸巾产品", "个人护理用纸")),
|
||||
("CHAIN_LIGHT_MID_PACKAGE_PAPER", ("纸包装产品", "纸包装", "瓦楞纸箱生产", "彩盒包装生产", "瓦楞包装制品", "瓦楞包装", "彩盒包装", "纸质包装", "纸制品包装", "纸制包装品")),
|
||||
("CHAIN_LIGHT_MID_PACKAGE_METAL", ("金属包装产品", "金属包装", "金属易拉罐", "三片罐", "二片罐", "塑料包装制品", "饮料罐生产", "包装容器生产")),
|
||||
("CHAIN_LIGHT_MID_FURNITURE_CUSTOM", ("定制家居", "定制橱柜", "定制衣柜", "全屋定制")),
|
||||
("CHAIN_LIGHT_MID_FURNITURE_SOFT", ("软体家具业务", "软体家具生产", "软体家具", "成品家具生产", "沙发生产", "沙发", "床垫生产", "床垫")),
|
||||
("CHAIN_LIGHT_MID_HARDWARE", ("家居五金产品", "家居五金业务", "家居五金", "卫浴产品", "智能马桶产品", "智能马桶生产", "智能马桶", "家具五金产品")),
|
||||
("CHAIN_LIGHT_MID_STATIONERY", ("文具产品", "办公文具", "学生文具", "文娱用品")),
|
||||
("CHAIN_LIGHT_DOWN_RETAIL", ("家居零售", "家居卖场", "整装服务", "家装服务")),
|
||||
("CHAIN_LIGHT_DOWN_EXPORT", ("家具出口", "家居用品出口", "家居跨境电商")),
|
||||
)
|
||||
@@ -0,0 +1,485 @@
|
||||
"""Industry definitions and product rules: energy industry."""
|
||||
|
||||
SOLID_STATE_BATTERY_DEFINITIONS = (
|
||||
("CHAIN_SSB", "固态电池产业链", "", 1),
|
||||
("CHAIN_SSB_UP", "上游材料", "CHAIN_SSB", 2),
|
||||
("CHAIN_SSB_UP_RESOURCE", "锂镍等资源", "CHAIN_SSB_UP", 3),
|
||||
("CHAIN_SSB_UP_CATHODE", "正极材料", "CHAIN_SSB_UP", 3),
|
||||
("CHAIN_SSB_UP_ANODE", "负极材料", "CHAIN_SSB_UP", 3),
|
||||
("CHAIN_SSB_UP_ELECTROLYTE", "固态电解质", "CHAIN_SSB_UP", 3),
|
||||
("CHAIN_SSB_UP_ELEC_SULFIDE", "硫化物电解质", "CHAIN_SSB_UP_ELECTROLYTE", 4),
|
||||
("CHAIN_SSB_UP_ELEC_OXIDE", "氧化物电解质", "CHAIN_SSB_UP_ELECTROLYTE", 4),
|
||||
("CHAIN_SSB_UP_ELEC_POLYMER", "聚合物电解质", "CHAIN_SSB_UP_ELECTROLYTE", 4),
|
||||
("CHAIN_SSB_UP_ELEC_HALIDE", "卤化物电解质", "CHAIN_SSB_UP_ELECTROLYTE", 4),
|
||||
("CHAIN_SSB_UP_AUXILIARY", "导电剂/集流体/辅材", "CHAIN_SSB_UP", 3),
|
||||
("CHAIN_SSB_MID", "中游制造", "CHAIN_SSB", 2),
|
||||
("CHAIN_SSB_MID_CELL", "固态/半固态电芯", "CHAIN_SSB_MID", 3),
|
||||
("CHAIN_SSB_MID_CELL_FULL", "全固态电芯", "CHAIN_SSB_MID_CELL", 4),
|
||||
("CHAIN_SSB_MID_CELL_SEMI", "半固态电芯", "CHAIN_SSB_MID_CELL", 4),
|
||||
("CHAIN_SSB_MID_EQUIPMENT", "电池制造设备", "CHAIN_SSB_MID", 3),
|
||||
("CHAIN_SSB_MID_EQ_ELECTRODE", "干法电极与涂布设备", "CHAIN_SSB_MID_EQUIPMENT", 4),
|
||||
("CHAIN_SSB_MID_EQ_ASSEMBLY", "叠片与装配设备", "CHAIN_SSB_MID_EQUIPMENT", 4),
|
||||
("CHAIN_SSB_MID_EQ_PRESS", "等静压与致密化设备", "CHAIN_SSB_MID_EQUIPMENT", 4),
|
||||
("CHAIN_SSB_MID_PACK", "PACK/BMS", "CHAIN_SSB_MID", 3),
|
||||
("CHAIN_SSB_DOWN", "下游应用", "CHAIN_SSB", 2),
|
||||
("CHAIN_SSB_DOWN_AUTO", "新能源汽车", "CHAIN_SSB_DOWN", 3),
|
||||
("CHAIN_SSB_DOWN_STORAGE", "储能", "CHAIN_SSB_DOWN", 3),
|
||||
("CHAIN_SSB_DOWN_LOW_ALTITUDE", "低空与航空", "CHAIN_SSB_DOWN", 3),
|
||||
("CHAIN_SSB_DOWN_CONSUMER", "消费电子", "CHAIN_SSB_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
SOLID_STATE_BATTERY_RULES = (
|
||||
("CHAIN_SSB_UP_ELEC_SULFIDE", ("硫化物电解质", "硫化物固态电解质", "硫化锂", "硫银锗矿型电解质")),
|
||||
("CHAIN_SSB_UP_ELEC_OXIDE", ("氧化物电解质", "锂镧锆氧", "llzo", "nasicon电解质")),
|
||||
("CHAIN_SSB_UP_ELEC_POLYMER", ("聚合物电解质", "固态聚合物电解质")),
|
||||
("CHAIN_SSB_UP_ELEC_HALIDE", ("卤化物电解质", "氯化物固态电解质")),
|
||||
("CHAIN_SSB_UP_ELECTROLYTE", ("固态电解质", "固态电池电解质", "固态电池电解质材料")),
|
||||
("CHAIN_SSB_UP_CATHODE", ("正极材料", "磷酸铁锂", "三元正极", "富锂锰基", "高镍正极", "钴酸锂")),
|
||||
("CHAIN_SSB_UP_ANODE", ("负极材料", "硅碳负极", "硅基负极", "锂金属负极", "人造石墨负极")),
|
||||
("CHAIN_SSB_UP_RESOURCE", ("锂矿", "碳酸锂", "氢氧化锂", "镍矿", "钴矿", "锂资源")),
|
||||
("CHAIN_SSB_UP_AUXILIARY", ("导电剂", "碳纳米管导电", "铜箔", "铝箔", "集流体", "电池粘结剂")),
|
||||
("CHAIN_SSB_MID_EQ_ELECTRODE", ("干法电极设备", "干法涂布", "固态电池涂布设备")),
|
||||
("CHAIN_SSB_MID_EQ_ASSEMBLY", ("固态电池叠片设备", "固态电池装配设备")),
|
||||
("CHAIN_SSB_MID_EQ_PRESS", ("等静压设备", "温等静压", "致密化设备")),
|
||||
("CHAIN_SSB_MID_EQUIPMENT", ("固态电池设备", "锂电池设备", "电池制造设备")),
|
||||
("CHAIN_SSB_MID_CELL_FULL", ("全固态电池", "全固态电芯")),
|
||||
("CHAIN_SSB_MID_CELL_SEMI", ("半固态电池", "固液混合电池", "半固态电芯")),
|
||||
("CHAIN_SSB_MID_CELL", ("固态锂电池", "固态电芯")),
|
||||
("CHAIN_SSB_MID_PACK", ("电池pack", "电池包", "电池管理系统", "bms", "动力电池系统")),
|
||||
("CHAIN_SSB_DOWN_AUTO", ("新能源汽车整车", "新能源乘用车", "电动汽车整车")),
|
||||
("CHAIN_SSB_DOWN_STORAGE", ("储能系统", "储能电站", "储能产品")),
|
||||
("CHAIN_SSB_DOWN_LOW_ALTITUDE", ("电动航空", "航空电池", "无人机电池", "低空飞行器")),
|
||||
("CHAIN_SSB_DOWN_CONSUMER", ("消费电子电池", "智能终端电池", "可穿戴设备电池")),
|
||||
)
|
||||
|
||||
|
||||
COAL_CHEMICAL_DEFINITIONS = (
|
||||
("CHAIN_COAL", "煤炭与煤化工产业链", "", 1),
|
||||
("CHAIN_COAL_UP", "上游资源与辅料", "CHAIN_COAL", 2),
|
||||
("CHAIN_COAL_UP_RESOURCE", "煤炭资源", "CHAIN_COAL_UP", 3),
|
||||
("CHAIN_COAL_UP_THERMAL", "动力煤", "CHAIN_COAL_UP_RESOURCE", 4),
|
||||
("CHAIN_COAL_UP_COKING", "焦煤", "CHAIN_COAL_UP_RESOURCE", 4),
|
||||
("CHAIN_COAL_UP_ANTHRACITE", "无烟煤", "CHAIN_COAL_UP_RESOURCE", 4),
|
||||
("CHAIN_COAL_UP_LIGNITE", "褐煤与其他煤种", "CHAIN_COAL_UP_RESOURCE", 4),
|
||||
("CHAIN_COAL_UP_WASH", "煤炭洗选", "CHAIN_COAL_UP", 3),
|
||||
("CHAIN_COAL_UP_EQUIPMENT", "煤机装备与矿服", "CHAIN_COAL_UP", 3),
|
||||
("CHAIN_COAL_UP_CATALYST", "催化剂与耐材", "CHAIN_COAL_UP", 3),
|
||||
("CHAIN_COAL_MID", "中游转化与工程", "CHAIN_COAL", 2),
|
||||
("CHAIN_COAL_MID_GASIFICATION", "煤气化技术与装备", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_MID_ENGINEERING", "煤化工工程服务", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_MID_COKE", "焦炭与焦化产品", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_MID_LIQUID", "煤制油/煤制气", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_MID_METHANOL", "甲醇/合成氨", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_MID_GAS", "工业气体", "CHAIN_COAL_MID", 3),
|
||||
("CHAIN_COAL_DOWN", "下游产品与应用", "CHAIN_COAL", 2),
|
||||
("CHAIN_COAL_DOWN_OLEFIN", "烯烃与精细化工", "CHAIN_COAL_DOWN", 3),
|
||||
("CHAIN_COAL_DOWN_FERTILIZER", "化肥", "CHAIN_COAL_DOWN", 3),
|
||||
("CHAIN_COAL_DOWN_ENERGY", "煤层气与能源利用", "CHAIN_COAL_DOWN", 3),
|
||||
("CHAIN_COAL_DOWN_LOGISTICS", "煤炭物流与贸易", "CHAIN_COAL_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
COAL_CHEMICAL_RULES = (
|
||||
("CHAIN_COAL_UP_THERMAL", ("动力煤开采", "动力煤生产", "动力煤产品", "电煤生产")),
|
||||
("CHAIN_COAL_UP_COKING", ("焦煤开采", "炼焦煤生产", "主焦煤", "配焦煤", "肥煤产品")),
|
||||
("CHAIN_COAL_UP_ANTHRACITE", ("无烟煤开采", "无烟煤生产", "无烟煤产品")),
|
||||
("CHAIN_COAL_UP_LIGNITE", ("褐煤开采", "褐煤生产", "长焰煤")),
|
||||
("CHAIN_COAL_UP_WASH", ("煤炭洗选业务", "原煤洗选", "洗精煤生产", "选煤服务")),
|
||||
("CHAIN_COAL_UP_EQUIPMENT", ("煤矿机械", "煤机装备", "煤炭采掘设备", "矿井智能化设备", "煤矿技术服务")),
|
||||
("CHAIN_COAL_MID_GASIFICATION", ("煤气化技术及关键设备", "煤气化技术", "煤气化设备", "气化炉", "煤气化炉", "粉煤气化", "水煤浆气化")),
|
||||
("CHAIN_COAL_MID_ENGINEERING", ("煤化工工程", "煤化工设计", "煤化工总承包", "煤化工技术服务", "煤化工设备成套")),
|
||||
("CHAIN_COAL_MID_COKE", ("焦炭生产", "冶金焦", "焦化产品生产", "炼焦业务", "煤焦油生产")),
|
||||
("CHAIN_COAL_MID_LIQUID", ("煤制油", "煤制天然气", "煤制气", "煤间接液化", "煤直接液化")),
|
||||
("CHAIN_COAL_MID_METHANOL", ("煤制甲醇", "甲醇生产", "合成氨", "煤制乙二醇")),
|
||||
("CHAIN_COAL_MID_GAS", ("工业气体运营", "工业气体", "合成气")),
|
||||
("CHAIN_COAL_UP_RESOURCE", ("煤炭开采", "煤炭生产", "煤炭资源")),
|
||||
("CHAIN_COAL_UP_CATALYST", ("煤化工催化剂", "气化炉耐火材料", "煤化工耐材")),
|
||||
("CHAIN_COAL_DOWN_OLEFIN", ("煤制烯烃", "煤基烯烃", "煤基聚烯烃", "煤基新材料")),
|
||||
("CHAIN_COAL_DOWN_FERTILIZER", ("尿素", "复合肥", "氮肥", "煤制化肥")),
|
||||
("CHAIN_COAL_DOWN_ENERGY", ("煤层气利用", "煤层气开发", "煤层气销售")),
|
||||
("CHAIN_COAL_DOWN_LOGISTICS", ("煤炭物流", "煤炭贸易", "煤炭供应链", "煤炭仓储", "煤炭铁路运输", "主焦煤供应链贸易服务", "非主焦煤供应链贸易服务", "焦煤供应链贸易")),
|
||||
)
|
||||
|
||||
|
||||
ENERGY_STORAGE_DEFINITIONS = (
|
||||
("CHAIN_ES", "新型储能产业链", "", 1),
|
||||
("CHAIN_ES_UP", "上游材料与部件", "CHAIN_ES", 2),
|
||||
("CHAIN_ES_UP_MATERIAL", "电池材料", "CHAIN_ES_UP", 3),
|
||||
("CHAIN_ES_UP_PCS", "PCS与功率器件", "CHAIN_ES_UP", 3),
|
||||
("CHAIN_ES_UP_CONTROL", "BMS/EMS与控制", "CHAIN_ES_UP", 3),
|
||||
("CHAIN_ES_MID", "储能技术与系统", "CHAIN_ES", 2),
|
||||
("CHAIN_ES_MID_LITHIUM", "锂离子储能", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_SODIUM", "钠离子储能", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_FLOW", "液流电池", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_PHYSICAL", "压缩空气/飞轮/重力储能", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_THERMAL", "储热与熔盐储能", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_SYSTEM", "储能系统集成", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_MID_THERMAL_CTRL", "热管理与消防", "CHAIN_ES_MID", 3),
|
||||
("CHAIN_ES_DOWN", "项目与应用", "CHAIN_ES", 2),
|
||||
("CHAIN_ES_DOWN_POWER", "电源侧储能", "CHAIN_ES_DOWN", 3),
|
||||
("CHAIN_ES_DOWN_GRID", "电网侧独立储能", "CHAIN_ES_DOWN", 3),
|
||||
("CHAIN_ES_DOWN_CI", "工商业储能", "CHAIN_ES_DOWN", 3),
|
||||
("CHAIN_ES_DOWN_HOME", "户用储能", "CHAIN_ES_DOWN", 3),
|
||||
("CHAIN_ES_DOWN_EPC", "储能EPC与运营", "CHAIN_ES_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
ENERGY_STORAGE_RULES = (
|
||||
("CHAIN_ES_UP_MATERIAL", ("储能电池材料", "储能正极材料", "储能负极材料", "储能电解液")),
|
||||
("CHAIN_ES_UP_PCS", ("储能变流器", "储能pcs", "储能逆变器", "双向变流器")),
|
||||
("CHAIN_ES_UP_CONTROL", ("储能bms", "储能ems", "储能能量管理系统", "储能控制系统")),
|
||||
("CHAIN_ES_MID_LITHIUM", ("锂离子储能电池", "磷酸铁锂储能", "储能锂电池")),
|
||||
("CHAIN_ES_MID_SODIUM", ("钠离子储能", "钠电储能", "钠离子电池储能")),
|
||||
("CHAIN_ES_MID_FLOW", ("全钒液流电池", "液流储能", "铁铬液流电池", "锌溴液流电池")),
|
||||
("CHAIN_ES_MID_PHYSICAL", ("压缩空气储能", "飞轮储能", "重力储能")),
|
||||
("CHAIN_ES_MID_THERMAL", ("熔盐储能", "储热系统", "相变储热")),
|
||||
("CHAIN_ES_MID_SYSTEM", ("储能系统集成", "储能集装箱", "储能一体柜", "储能系统解决方案")),
|
||||
("CHAIN_ES_MID_THERMAL_CTRL", ("储能热管理", "储能液冷", "储能消防", "储能温控")),
|
||||
("CHAIN_ES_DOWN_POWER", ("新能源配储", "风光储一体化", "电源侧储能")),
|
||||
("CHAIN_ES_DOWN_GRID", ("独立储能电站", "电网侧储能", "共享储能电站")),
|
||||
("CHAIN_ES_DOWN_CI", ("工商业储能", "用户侧储能", "园区储能")),
|
||||
("CHAIN_ES_DOWN_HOME", ("户用储能", "家庭储能", "便携式储能")),
|
||||
("CHAIN_ES_DOWN_EPC", ("储能电站epc", "储能项目运营", "储能电站运维")),
|
||||
)
|
||||
|
||||
|
||||
PV_DEFINITIONS = (
|
||||
("CHAIN_PV", "光伏产业链", "", 1),
|
||||
("CHAIN_PV_UP", "上游材料与设备", "CHAIN_PV", 2),
|
||||
("CHAIN_PV_UP_SILICON", "工业硅与多晶硅", "CHAIN_PV_UP", 3),
|
||||
("CHAIN_PV_UP_EQUIPMENT", "光伏制造设备", "CHAIN_PV_UP", 3),
|
||||
("CHAIN_PV_MID", "中游制造", "CHAIN_PV", 2),
|
||||
("CHAIN_PV_MID_WAFER", "硅片", "CHAIN_PV_MID", 3),
|
||||
("CHAIN_PV_MID_CELL", "光伏电池片", "CHAIN_PV_MID", 3),
|
||||
("CHAIN_PV_MID_CELL_TOPCON", "TOPCon", "CHAIN_PV_MID_CELL", 4),
|
||||
("CHAIN_PV_MID_CELL_HJT", "HJT", "CHAIN_PV_MID_CELL", 4),
|
||||
("CHAIN_PV_MID_CELL_BC", "BC电池", "CHAIN_PV_MID_CELL", 4),
|
||||
("CHAIN_PV_MID_CELL_PEROV", "钙钛矿与叠层电池", "CHAIN_PV_MID_CELL", 4),
|
||||
("CHAIN_PV_MID_MODULE", "光伏组件", "CHAIN_PV_MID", 3),
|
||||
("CHAIN_PV_MID_AUX", "玻璃/胶膜/银浆/背板", "CHAIN_PV_MID", 3),
|
||||
("CHAIN_PV_MID_INVERTER", "逆变器与支架", "CHAIN_PV_MID", 3),
|
||||
("CHAIN_PV_DOWN", "下游电站与应用", "CHAIN_PV", 2),
|
||||
("CHAIN_PV_DOWN_EPC", "光伏电站EPC", "CHAIN_PV_DOWN", 3),
|
||||
("CHAIN_PV_DOWN_OPS", "电站运营", "CHAIN_PV_DOWN", 3),
|
||||
("CHAIN_PV_DOWN_DISTRIBUTED", "分布式光伏与BIPV", "CHAIN_PV_DOWN", 3),
|
||||
("CHAIN_PV_DOWN_SPACE", "太空光伏", "CHAIN_PV_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
PV_RULES = (
|
||||
("CHAIN_PV_UP_SILICON", ("光伏多晶硅", "太阳能级多晶硅", "工业硅光伏", "硅料生产")),
|
||||
("CHAIN_PV_UP_EQUIPMENT", ("光伏设备", "单晶炉", "切片机", "光伏电池设备", "组件设备")),
|
||||
("CHAIN_PV_MID_WAFER", ("光伏硅片", "单晶硅片", "太阳能硅片")),
|
||||
("CHAIN_PV_MID_CELL_TOPCON", ("topcon电池", "topcon光伏")),
|
||||
("CHAIN_PV_MID_CELL_HJT", ("hjt电池", "异质结电池")),
|
||||
("CHAIN_PV_MID_CELL_BC", ("bc电池", "ibc电池", "背接触电池")),
|
||||
("CHAIN_PV_MID_CELL_PEROV", ("钙钛矿电池", "钙钛矿叠层", "晶硅叠层电池")),
|
||||
("CHAIN_PV_MID_CELL", ("光伏电池片", "太阳能电池片")),
|
||||
("CHAIN_PV_MID_MODULE", ("光伏组件", "太阳能组件")),
|
||||
("CHAIN_PV_MID_AUX", ("光伏玻璃", "光伏胶膜", "光伏银浆", "光伏背板", "光伏焊带", "光伏边框")),
|
||||
("CHAIN_PV_MID_INVERTER", ("光伏逆变器", "光伏支架", "跟踪支架")),
|
||||
("CHAIN_PV_DOWN_EPC", ("光伏电站epc", "光伏工程总承包", "光伏电站建设")),
|
||||
("CHAIN_PV_DOWN_OPS", ("光伏电站运营", "太阳能发电运营")),
|
||||
("CHAIN_PV_DOWN_DISTRIBUTED", ("分布式光伏", "户用光伏", "bipv", "光伏建筑一体化")),
|
||||
("CHAIN_PV_DOWN_SPACE", ("太空光伏", "空间太阳能电池", "航天光伏")),
|
||||
)
|
||||
|
||||
|
||||
WIND_DEFINITIONS = (
|
||||
("CHAIN_WIND", "风电产业链", "", 1),
|
||||
("CHAIN_WIND_UP", "上游材料", "CHAIN_WIND", 2),
|
||||
("CHAIN_WIND_UP_STEEL", "钢材/铸锻件/稀土磁材", "CHAIN_WIND_UP", 3),
|
||||
("CHAIN_WIND_UP_COMPOSITE", "叶片复合材料", "CHAIN_WIND_UP", 3),
|
||||
("CHAIN_WIND_MID", "零部件与整机", "CHAIN_WIND", 2),
|
||||
("CHAIN_WIND_MID_BLADE", "叶片", "CHAIN_WIND_MID", 3),
|
||||
("CHAIN_WIND_MID_DRIVE", "齿轮箱/主轴/轴承", "CHAIN_WIND_MID", 3),
|
||||
("CHAIN_WIND_MID_ELECTRIC", "发电机/变流器/控制", "CHAIN_WIND_MID", 3),
|
||||
("CHAIN_WIND_MID_TOWER", "塔筒/桩基/海缆", "CHAIN_WIND_MID", 3),
|
||||
("CHAIN_WIND_MID_TURBINE", "风机整机", "CHAIN_WIND_MID", 3),
|
||||
("CHAIN_WIND_MID_ONSHORE", "陆上风机", "CHAIN_WIND_MID_TURBINE", 4),
|
||||
("CHAIN_WIND_MID_OFFSHORE", "海上风机", "CHAIN_WIND_MID_TURBINE", 4),
|
||||
("CHAIN_WIND_DOWN", "开发建设与运营", "CHAIN_WIND", 2),
|
||||
("CHAIN_WIND_DOWN_DEV", "风电场开发", "CHAIN_WIND_DOWN", 3),
|
||||
("CHAIN_WIND_DOWN_EPC", "吊装施工与EPC", "CHAIN_WIND_DOWN", 3),
|
||||
("CHAIN_WIND_DOWN_OPS", "风电运营与运维", "CHAIN_WIND_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
WIND_RULES = (
|
||||
("CHAIN_WIND_UP_STEEL", ("风电铸件", "风电锻件", "风电钢材", "风电稀土磁材")),
|
||||
("CHAIN_WIND_UP_COMPOSITE", ("风电碳纤维", "叶片树脂", "叶片复合材料", "风电玻纤")),
|
||||
("CHAIN_WIND_MID_BLADE", ("风电叶片", "风机叶片")),
|
||||
("CHAIN_WIND_MID_DRIVE", ("风电齿轮箱", "风电主轴", "风电轴承", "风机轴承")),
|
||||
("CHAIN_WIND_MID_ELECTRIC", ("风力发电机", "风电变流器", "风机控制系统")),
|
||||
("CHAIN_WIND_MID_TOWER", ("风电塔筒", "海上风电桩基", "风电海缆", "风电基础")),
|
||||
("CHAIN_WIND_MID_OFFSHORE", ("海上风机整机", "海上风电整机")),
|
||||
("CHAIN_WIND_MID_ONSHORE", ("陆上风机整机", "陆上风电整机")),
|
||||
("CHAIN_WIND_MID_TURBINE", ("风电整机", "风力发电机组")),
|
||||
("CHAIN_WIND_DOWN_DEV", ("风电场开发", "海上风电开发", "陆上风电开发")),
|
||||
("CHAIN_WIND_DOWN_EPC", ("风电吊装", "风电工程epc", "风电场建设", "海上风电施工")),
|
||||
("CHAIN_WIND_DOWN_OPS", ("风电场运营", "风电运维", "风机运维")),
|
||||
)
|
||||
|
||||
|
||||
HYDROGEN_DEFINITIONS = (
|
||||
("CHAIN_H2", "氢能产业链", "", 1),
|
||||
("CHAIN_H2_UP", "制氢", "CHAIN_H2", 2),
|
||||
("CHAIN_H2_UP_ALK", "碱性电解槽", "CHAIN_H2_UP", 3),
|
||||
("CHAIN_H2_UP_PEM", "PEM/AEM/SOEC电解槽", "CHAIN_H2_UP", 3),
|
||||
("CHAIN_H2_UP_INDUSTRIAL", "工业副产氢与化石制氢", "CHAIN_H2_UP", 3),
|
||||
("CHAIN_H2_MID", "储运与加注", "CHAIN_H2", 2),
|
||||
("CHAIN_H2_MID_COMPRESS", "压缩机与储氢瓶", "CHAIN_H2_MID", 3),
|
||||
("CHAIN_H2_MID_LIQUID", "液氢/管道/运氢", "CHAIN_H2_MID", 3),
|
||||
("CHAIN_H2_MID_STATION", "加氢站", "CHAIN_H2_MID", 3),
|
||||
("CHAIN_H2_DOWN", "转化与应用", "CHAIN_H2", 2),
|
||||
("CHAIN_H2_DOWN_FC", "燃料电池系统", "CHAIN_H2_DOWN", 3),
|
||||
("CHAIN_H2_DOWN_STACK", "电堆与关键材料", "CHAIN_H2_DOWN", 3),
|
||||
("CHAIN_H2_DOWN_VEHICLE", "氢能交通", "CHAIN_H2_DOWN", 3),
|
||||
("CHAIN_H2_DOWN_INDUSTRY", "工业脱碳与氢冶金", "CHAIN_H2_DOWN", 3),
|
||||
("CHAIN_H2_DOWN_FUEL", "绿色甲醇/绿氨/航煤", "CHAIN_H2_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
HYDROGEN_RULES = (
|
||||
("CHAIN_H2_UP_ALK", ("碱性电解槽", "碱性水电解制氢", "alk电解槽")),
|
||||
("CHAIN_H2_UP_PEM", ("pem电解槽", "aem电解槽", "soec电解槽", "质子交换膜电解槽")),
|
||||
("CHAIN_H2_UP_INDUSTRIAL", ("工业副产氢", "工业尾气提纯高纯氢", "工业尾气回收提纯高纯氢", "高纯氢气生产", "煤制氢", "天然气制氢", "蓝氢")),
|
||||
("CHAIN_H2_MID_COMPRESS", ("氢气压缩机", "储氢瓶", "储氢容器", "高压储氢")),
|
||||
("CHAIN_H2_MID_LIQUID", ("液氢", "输氢管道", "氢气运输", "运氢装备")),
|
||||
("CHAIN_H2_MID_STATION", ("加氢站", "加氢机", "加氢设备")),
|
||||
("CHAIN_H2_DOWN_STACK", ("燃料电池电堆", "质子交换膜", "膜电极", "双极板", "燃料电池催化剂")),
|
||||
("CHAIN_H2_DOWN_FC", ("氢燃料电池系统", "燃料电池发动机", "燃料电池系统")),
|
||||
("CHAIN_H2_DOWN_VEHICLE", ("氢燃料电池汽车", "氢能重卡", "氢能船舶")),
|
||||
("CHAIN_H2_DOWN_INDUSTRY", ("氢冶金", "绿氢炼化", "绿氢化工", "工业绿氢")),
|
||||
("CHAIN_H2_DOWN_FUEL", ("绿色甲醇", "绿氨", "绿色航煤", "可持续航空燃料")),
|
||||
)
|
||||
|
||||
|
||||
NEW_ENERGY_AUTO_DEFINITIONS = (
|
||||
("CHAIN_AUTO", "新能源汽车产业链", "", 1),
|
||||
("CHAIN_AUTO_UP", "核心系统与零部件", "CHAIN_AUTO", 2),
|
||||
("CHAIN_AUTO_UP_BATTERY", "动力电池系统", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_BATTERY_CELL", "动力电池与电芯", "CHAIN_AUTO_UP_BATTERY", 4),
|
||||
("CHAIN_AUTO_UP_BATTERY_PACK", "PACK/BMS", "CHAIN_AUTO_UP_BATTERY", 4),
|
||||
("CHAIN_AUTO_UP_DRIVE", "电驱动系统", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_DRIVE_MOTOR", "驱动电机", "CHAIN_AUTO_UP_DRIVE", 4),
|
||||
("CHAIN_AUTO_UP_DRIVE_CONTROL", "电控/功率模块", "CHAIN_AUTO_UP_DRIVE", 4),
|
||||
("CHAIN_AUTO_UP_DRIVE_REDUCER", "减速器与电驱总成", "CHAIN_AUTO_UP_DRIVE", 4),
|
||||
("CHAIN_AUTO_UP_THERMAL", "热管理", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_SMART", "智能驾驶", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_SMART_SENSOR", "摄像头/雷达/传感器", "CHAIN_AUTO_UP_SMART", 4),
|
||||
("CHAIN_AUTO_UP_SMART_COMPUTE", "智驾芯片与域控制器", "CHAIN_AUTO_UP_SMART", 4),
|
||||
("CHAIN_AUTO_UP_SMART_SOFTWARE", "智能驾驶软件", "CHAIN_AUTO_UP_SMART", 4),
|
||||
("CHAIN_AUTO_UP_ELECTRONIC", "座舱与汽车电子", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_INTERIOR", "汽车内饰与功能材料", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_CHASSIS", "车身与智能底盘", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_UP_CHARGE", "充换电设备", "CHAIN_AUTO_UP", 3),
|
||||
("CHAIN_AUTO_MID", "整车制造", "CHAIN_AUTO", 2),
|
||||
("CHAIN_AUTO_MID_PASSENGER", "新能源乘用车", "CHAIN_AUTO_MID", 3),
|
||||
("CHAIN_AUTO_MID_COMMERCIAL", "新能源商用车", "CHAIN_AUTO_MID", 3),
|
||||
("CHAIN_AUTO_MID_SPECIAL", "专用车与两轮车", "CHAIN_AUTO_MID", 3),
|
||||
("CHAIN_AUTO_DOWN", "销售服务与后市场", "CHAIN_AUTO", 2),
|
||||
("CHAIN_AUTO_DOWN_SALES", "汽车销售与出行服务", "CHAIN_AUTO_DOWN", 3),
|
||||
("CHAIN_AUTO_DOWN_CHARGE", "充换电运营", "CHAIN_AUTO_DOWN", 3),
|
||||
("CHAIN_AUTO_DOWN_RECYCLE", "动力电池回收", "CHAIN_AUTO_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
NEW_ENERGY_AUTO_RULES = (
|
||||
("CHAIN_AUTO_UP_BATTERY_CELL", ("动力电池电芯", "新能源汽车动力电池", "车用锂离子电池", "动力电池研发")),
|
||||
("CHAIN_AUTO_UP_BATTERY_PACK", ("动力电池pack", "动力电池包", "动力电池bms", "电池管理系统")),
|
||||
("CHAIN_AUTO_UP_DRIVE_MOTOR", ("新能源汽车驱动电机", "永磁同步驱动电机", "扁线电机", "轮毂电机")),
|
||||
("CHAIN_AUTO_UP_DRIVE_CONTROL", ("新能源汽车电控", "电机控制器", "车用功率模块", "车规级igbt模块", "电驱控制器")),
|
||||
("CHAIN_AUTO_UP_DRIVE_REDUCER", ("新能源汽车减速器", "电驱动总成", "三合一电驱", "多合一电驱")),
|
||||
("CHAIN_AUTO_UP_THERMAL", ("新能源汽车热管理", "汽车热管理系统", "电动压缩机", "电子水泵", "热泵空调", "电池冷却板")),
|
||||
("CHAIN_AUTO_UP_SMART_SENSOR", ("车载摄像头", "车载激光雷达", "车载毫米波雷达", "自动驾驶传感器")),
|
||||
("CHAIN_AUTO_UP_SMART_COMPUTE", ("自动驾驶域控制器", "智能驾驶域控制器", "车载计算平台", "智能驾驶芯片")),
|
||||
("CHAIN_AUTO_UP_SMART_SOFTWARE", ("自动驾驶软件", "智能驾驶算法", "高阶辅助驾驶系统", "adas软件")),
|
||||
("CHAIN_AUTO_UP_ELECTRONIC", ("智能座舱", "车载信息娱乐系统", "车载显示", "汽车电子控制单元", "车载终端")),
|
||||
("CHAIN_AUTO_UP_INTERIOR", ("汽车内饰用皮革", "汽车内饰皮革", "新能源汽车内饰", "汽车内饰材料")),
|
||||
("CHAIN_AUTO_UP_CHASSIS", ("线控底盘", "线控制动", "线控转向", "空气悬架", "一体化压铸", "汽车轻量化结构件")),
|
||||
("CHAIN_AUTO_UP_CHARGE", ("新能源汽车充电桩", "换电设备", "车载充电机", "充配电系统")),
|
||||
("CHAIN_AUTO_MID_PASSENGER", ("新能源乘用车", "纯电动乘用车", "插电式混合动力乘用车")),
|
||||
("CHAIN_AUTO_MID_COMMERCIAL", ("新能源商用车", "新能源客车", "新能源重卡", "电动物流车")),
|
||||
("CHAIN_AUTO_MID_SPECIAL", ("新能源专用车", "电动两轮车", "电动摩托车")),
|
||||
("CHAIN_AUTO_DOWN_SALES", ("新能源汽车销售", "新能源汽车经销", "新能源汽车出行服务")),
|
||||
("CHAIN_AUTO_DOWN_CHARGE", ("充电站运营", "换电站运营", "新能源汽车充换电运营")),
|
||||
("CHAIN_AUTO_DOWN_RECYCLE", ("动力电池回收", "退役动力电池", "动力电池梯次利用")),
|
||||
)
|
||||
|
||||
|
||||
PETROCHEMICAL_DEFINITIONS = (
|
||||
("CHAIN_CHEM", "石化与精细化工产业链", "", 1),
|
||||
("CHAIN_CHEM_UP", "油气资源与服务", "CHAIN_CHEM", 2),
|
||||
("CHAIN_CHEM_UP_EXPLORE", "油气勘探开采", "CHAIN_CHEM_UP", 3),
|
||||
("CHAIN_CHEM_UP_SERVICE", "油田服务与工程", "CHAIN_CHEM_UP", 3),
|
||||
("CHAIN_CHEM_UP_STORAGE", "油气储运", "CHAIN_CHEM_UP", 3),
|
||||
("CHAIN_CHEM_MID", "炼化与基础化工", "CHAIN_CHEM", 2),
|
||||
("CHAIN_CHEM_MID_REFINING", "炼油与成品油", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_OLEFIN", "烯烃产业链", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_AROMATIC", "芳烃产业链", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_BASIC", "基础无机化工", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_RESIN", "合成树脂与塑料", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_RUBBER", "合成橡胶", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_MID_FIBER", "化学纤维", "CHAIN_CHEM_MID", 3),
|
||||
("CHAIN_CHEM_FINE", "精细化工", "CHAIN_CHEM", 2),
|
||||
("CHAIN_CHEM_FINE_ELECTRONIC", "电子化学品", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_FINE_ADDITIVE", "催化剂与助剂", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_FINE_COATING", "涂料染料与油墨", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_FINE_AGRO", "农药与农化", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_FINE_FLUORINE", "氟硅化工", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_FINE_SPECIAL", "特种与功能化学品", "CHAIN_CHEM_FINE", 3),
|
||||
("CHAIN_CHEM_DOWN", "流通与终端材料", "CHAIN_CHEM", 2),
|
||||
("CHAIN_CHEM_DOWN_TRADE", "化工品贸易与分销", "CHAIN_CHEM_DOWN", 3),
|
||||
("CHAIN_CHEM_DOWN_MATERIAL", "高分子与复合材料", "CHAIN_CHEM_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
PETROCHEMICAL_RULES = (
|
||||
("CHAIN_CHEM_UP_EXPLORE", ("石油勘探开发", "天然气勘探开发", "油气开采", "原油开采", "页岩气开采")),
|
||||
("CHAIN_CHEM_UP_SERVICE", ("油田技术服务", "油气田工程", "钻井服务", "测井服务", "油服装备")),
|
||||
("CHAIN_CHEM_UP_STORAGE", ("原油储运", "成品油储运", "天然气管道运输", "油气仓储")),
|
||||
("CHAIN_CHEM_MID_REFINING", ("原油炼制", "石油炼化", "炼油业务", "成品油生产")),
|
||||
("CHAIN_CHEM_MID_OLEFIN", ("乙烯生产", "丙烯生产", "烯烃项目", "煤制烯烃", "丙烷脱氢")),
|
||||
("CHAIN_CHEM_MID_AROMATIC", ("芳烃生产", "对二甲苯", "px生产", "pta生产", "纯苯生产")),
|
||||
("CHAIN_CHEM_MID_BASIC", ("氯碱化工", "纯碱生产", "硫酸生产", "烧碱生产", "无机化工原料")),
|
||||
("CHAIN_CHEM_MID_RESIN", ("聚乙烯树脂", "聚丙烯树脂", "工程塑料", "合成树脂", "改性塑料")),
|
||||
("CHAIN_CHEM_MID_RUBBER", ("合成橡胶", "丁苯橡胶", "顺丁橡胶", "特种橡胶")),
|
||||
("CHAIN_CHEM_MID_FIBER", ("化学纤维", "涤纶长丝", "锦纶", "氨纶", "粘胶纤维")),
|
||||
("CHAIN_CHEM_FINE_ELECTRONIC", ("电子化学品", "湿电子化学品", "半导体化学品", "显示面板化学品", "电子级试剂")),
|
||||
("CHAIN_CHEM_FINE_ADDITIVE", ("化工催化剂", "塑料助剂", "橡胶助剂", "高分子材料防老化助剂", "受阻胺光稳定剂", "光稳定剂", "阻聚剂", "表面活性剂", "化工添加剂")),
|
||||
("CHAIN_CHEM_FINE_COATING", ("工业涂料", "精细染料", "印刷油墨", "有机颜料")),
|
||||
("CHAIN_CHEM_FINE_AGRO", ("农药原药", "农药制剂", "除草剂", "杀虫剂", "杀菌剂")),
|
||||
("CHAIN_CHEM_FINE_FLUORINE", ("氟化工产品", "含氟聚合物", "制冷剂", "有机硅材料", "硅烷偶联剂")),
|
||||
("CHAIN_CHEM_FINE_SPECIAL", ("特种化学品", "功能化学品", "专用化学品", "高纯化学品")),
|
||||
("CHAIN_CHEM_DOWN_TRADE", ("化工品贸易", "石化产品贸易", "化学品分销")),
|
||||
("CHAIN_CHEM_DOWN_MATERIAL", ("高分子复合材料", "功能膜材料", "特种复合材料", "高性能聚合物材料")),
|
||||
)
|
||||
|
||||
|
||||
STEEL_DEFINITIONS = (
|
||||
("CHAIN_STEEL", "钢铁产业链", "", 1),
|
||||
("CHAIN_STEEL_UP", "上游原料与冶炼", "CHAIN_STEEL", 2),
|
||||
("CHAIN_STEEL_UP_IRON", "铁矿石采选与球团", "CHAIN_STEEL_UP", 3),
|
||||
("CHAIN_STEEL_UP_COKE", "焦化与炉料", "CHAIN_STEEL_UP", 3),
|
||||
("CHAIN_STEEL_UP_SMELT", "钢铁冶炼", "CHAIN_STEEL_UP", 3),
|
||||
("CHAIN_STEEL_MID", "中游钢材制造", "CHAIN_STEEL", 2),
|
||||
("CHAIN_STEEL_MID_INTEGRATED", "综合钢铁制造", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_MID_LONG", "长材", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_MID_FLAT", "板材", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_MID_PIPE", "钢管", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_MID_STAINLESS", "不锈钢", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_MID_SPECIAL", "特钢与高温合金", "CHAIN_STEEL_MID", 3),
|
||||
("CHAIN_STEEL_DOWN", "下游加工与流通", "CHAIN_STEEL", 2),
|
||||
("CHAIN_STEEL_DOWN_PROCESS", "钢材深加工", "CHAIN_STEEL_DOWN", 3),
|
||||
("CHAIN_STEEL_DOWN_DISTRIBUTE", "钢材贸易与配送", "CHAIN_STEEL_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
STEEL_RULES = (
|
||||
("CHAIN_STEEL_UP_IRON", ("铁矿石开采", "铁矿石采选", "铁精粉生产", "球团矿生产")),
|
||||
("CHAIN_STEEL_UP_COKE", ("焦炭生产", "焦化产品", "冶金焦", "球团矿生产")),
|
||||
("CHAIN_STEEL_UP_SMELT", ("钢铁冶炼", "炼铁炼钢", "钢坯生产", "粗钢生产")),
|
||||
("CHAIN_STEEL_MID_LONG", ("螺纹钢生产", "线材生产", "棒材生产", "型钢生产", "建筑钢材")),
|
||||
("CHAIN_STEEL_MID_FLAT", ("热轧板卷", "冷轧板卷", "中厚板生产", "钢板生产", "硅钢产品", "汽车用钢板", "汽车钢板", "电工钢")),
|
||||
("CHAIN_STEEL_MID_PIPE", ("加工、生产和销售离心球墨铸铁管", "无缝钢管生产", "无缝管", "焊接钢管", "球墨铸铁管", "油井管", "钢管产品")),
|
||||
("CHAIN_STEEL_MID_STAINLESS", ("工业用不锈钢管", "不锈钢冶炼", "不锈钢板材", "不锈钢管", "不锈钢产品")),
|
||||
("CHAIN_STEEL_MID_SPECIAL", ("特殊钢生产", "特殊钢", "优特钢", "特种钢材", "高温合金", "工模具钢", "轴承钢", "齿轮钢", "弹簧扁钢", "优线")),
|
||||
("CHAIN_STEEL_DOWN_PROCESS", ("钢材深加工", "钢结构加工", "钢板剪切加工", "金属压延加工")),
|
||||
("CHAIN_STEEL_DOWN_DISTRIBUTE", ("钢材贸易", "钢材仓储配送", "钢铁供应链服务")),
|
||||
("CHAIN_STEEL_MID_INTEGRATED", ("钢铁与钢铁主业相关", "钢铁冶金产品", "钢铁冶炼、轧制", "黑色金属冶炼", "钢铁产品", "钢材产品", "钢材生产", "钢铁及其压延产品")),
|
||||
)
|
||||
|
||||
|
||||
NONFERROUS_DEFINITIONS = (
|
||||
("CHAIN_METAL", "有色金属产业链", "", 1),
|
||||
("CHAIN_METAL_COPPER", "铜产业链", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_COPPER_MINE", "铜矿采选与冶炼", "CHAIN_METAL_COPPER", 3),
|
||||
("CHAIN_METAL_COPPER_PROCESS", "铜材加工", "CHAIN_METAL_COPPER", 3),
|
||||
("CHAIN_METAL_AL", "铝产业链", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_AL_SMELT", "铝土矿氧化铝与电解铝", "CHAIN_METAL_AL", 3),
|
||||
("CHAIN_METAL_AL_PROCESS", "铝材加工", "CHAIN_METAL_AL", 3),
|
||||
("CHAIN_METAL_LEAD_ZINC", "铅锌产业链", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_LEAD_ZINC_MINE", "铅锌采选与冶炼", "CHAIN_METAL_LEAD_ZINC", 3),
|
||||
("CHAIN_METAL_LEAD_ZINC_PROCESS", "铅锌加工", "CHAIN_METAL_LEAD_ZINC", 3),
|
||||
("CHAIN_METAL_ENERGY", "能源金属", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_ENERGY_LI", "锂资源与锂盐", "CHAIN_METAL_ENERGY", 3),
|
||||
("CHAIN_METAL_ENERGY_NI_CO", "镍钴资源与冶炼", "CHAIN_METAL_ENERGY", 3),
|
||||
("CHAIN_METAL_RARE", "稀土与战略小金属", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_RARE_EARTH", "稀土采选冶炼与磁材", "CHAIN_METAL_RARE", 3),
|
||||
("CHAIN_METAL_RARE_W_MO", "钨钼", "CHAIN_METAL_RARE", 3),
|
||||
("CHAIN_METAL_RARE_OTHER", "锡锑铟锗等小金属", "CHAIN_METAL_RARE", 3),
|
||||
("CHAIN_METAL_PRECIOUS", "贵金属", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_PRECIOUS_GOLD", "黄金", "CHAIN_METAL_PRECIOUS", 3),
|
||||
("CHAIN_METAL_PRECIOUS_SILVER", "白银与铂族金属", "CHAIN_METAL_PRECIOUS", 3),
|
||||
("CHAIN_METAL_NEW", "金属新材料与循环", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_NEW_TITANIUM", "钛镁与轻合金", "CHAIN_METAL_NEW", 3),
|
||||
("CHAIN_METAL_NEW_POWDER", "金属粉体与功能材料", "CHAIN_METAL_NEW", 3),
|
||||
("CHAIN_METAL_NEW_RECYCLE", "再生有色金属", "CHAIN_METAL_NEW", 3),
|
||||
("CHAIN_METAL_MULTI", "综合矿业与多金属", "CHAIN_METAL", 2),
|
||||
("CHAIN_METAL_MULTI_MINE", "多金属采选", "CHAIN_METAL_MULTI", 3),
|
||||
("CHAIN_METAL_MULTI_SMELT", "综合有色冶炼加工", "CHAIN_METAL_MULTI", 3),
|
||||
)
|
||||
|
||||
|
||||
NONFERROUS_RULES = (
|
||||
("CHAIN_METAL_COPPER_MINE", ("铜矿采选", "铜精矿", "阴极铜冶炼", "电解铜生产")),
|
||||
("CHAIN_METAL_COPPER_PROCESS", ("铜板带箔", "铜箔生产", "铜杆生产", "铜管生产", "铜合金加工")),
|
||||
("CHAIN_METAL_AL_SMELT", ("铝土矿开采", "氧化铝生产", "电解铝生产", "原铝生产")),
|
||||
("CHAIN_METAL_AL_PROCESS", ("铝板带箔", "铝型材生产", "铝合金加工", "铝压延产品")),
|
||||
("CHAIN_METAL_LEAD_ZINC_MINE", ("铅锌等有色金属的采选、冶炼", "铅锌金属的冶炼", "铅锌矿采选", "铅精矿", "锌精矿", "锌锭", "电解锌生产", "铅冶炼")),
|
||||
("CHAIN_METAL_LEAD_ZINC_PROCESS", ("锌合金产品", "压铸锌合金", "铅合金加工")),
|
||||
("CHAIN_METAL_ENERGY_LI", ("锂矿采选", "锂辉石精矿", "碳酸锂生产", "氢氧化锂生产", "锂盐产品")),
|
||||
("CHAIN_METAL_ENERGY_NI_CO", ("镍矿采选", "镍冶炼", "电解镍", "钴矿采选", "钴冶炼", "钴盐生产")),
|
||||
("CHAIN_METAL_RARE_EARTH", ("稀土矿采选", "稀土冶炼分离", "稀土永磁材料", "钕铁硼磁材")),
|
||||
("CHAIN_METAL_RARE_W_MO", ("钨矿采选", "钨冶炼", "硬质合金", "钼矿采选", "钼冶炼", "钼制品")),
|
||||
("CHAIN_METAL_RARE_OTHER", ("锡矿采选", "锡冶炼", "锑矿采选", "锑制品", "铟锭生产", "锗产品")),
|
||||
("CHAIN_METAL_PRECIOUS_GOLD", ("贵金属和有色金属矿采选及金属贸易", "黄金矿山", "金矿采选", "黄金冶炼", "合质金", "黄金贸易", "标准金锭")),
|
||||
("CHAIN_METAL_PRECIOUS_SILVER", ("白银冶炼", "银锭生产", "铂族金属", "铂钯产品")),
|
||||
("CHAIN_METAL_NEW_TITANIUM", ("海绵钛生产", "钛材加工", "钛合金材料", "镁合金产品", "镁铝合金压铸", "镁合金生产", "镁合金加工")),
|
||||
("CHAIN_METAL_NEW_POWDER", ("金属粉末生产", "金属粉体材料", "非晶合金材料")),
|
||||
("CHAIN_METAL_NEW_RECYCLE", ("再生铝生产", "再生铜生产", "废旧有色金属回收", "再生有色金属")),
|
||||
("CHAIN_METAL_MULTI_MINE", ("多种有色金属及贵金属的采选、冶炼", "有色金属矿采选", "有色金属采选", "多金属矿采选", "贵金属和有色金属矿采选")),
|
||||
("CHAIN_METAL_MULTI_SMELT", ("有色金属冶炼加工", "有色金属冶炼", "有色金属压延加工")),
|
||||
)
|
||||
|
||||
|
||||
AUTO_AFTERMARKET_DEFINITIONS = (
|
||||
("CHAIN_AF", "汽车后市场产业链", "", 1),
|
||||
("CHAIN_AF_UP", "上游替换件与耗材", "CHAIN_AF", 2),
|
||||
("CHAIN_AF_UP_TIRE", "轮胎", "CHAIN_AF_UP", 3),
|
||||
("CHAIN_AF_UP_BATTERY", "汽车蓄电池", "CHAIN_AF_UP", 3),
|
||||
("CHAIN_AF_UP_FLUID", "润滑油与养护品", "CHAIN_AF_UP", 3),
|
||||
("CHAIN_AF_UP_PART", "维修替换零部件", "CHAIN_AF_UP", 3),
|
||||
("CHAIN_AF_MID", "中游销售与流通", "CHAIN_AF", 2),
|
||||
("CHAIN_AF_MID_DEALER", "汽车经销与4S店", "CHAIN_AF_MID", 3),
|
||||
("CHAIN_AF_MID_PART", "汽配流通与供应链", "CHAIN_AF_MID", 3),
|
||||
("CHAIN_AF_MID_PLATFORM", "汽车交易平台", "CHAIN_AF_MID", 3),
|
||||
("CHAIN_AF_DOWN", "下游使用服务", "CHAIN_AF", 2),
|
||||
("CHAIN_AF_DOWN_REPAIR", "维修保养与美容", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_TEST", "检测与认证", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_USED", "二手车交易", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_RECYCLE", "拆解与回收", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_RENT", "租赁与融资租赁", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_ROADSIDE", "道路救援与保险服务", "CHAIN_AF_DOWN", 3),
|
||||
("CHAIN_AF_DOWN_MOBILITY", "网约车与出行运营", "CHAIN_AF_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
AUTO_AFTERMARKET_RULES = (
|
||||
("CHAIN_AF_UP_TIRE", ("汽车轮胎生产", "乘用车轮胎", "商用车轮胎", "全钢子午线轮胎", "半钢子午线轮胎")),
|
||||
("CHAIN_AF_UP_BATTERY", ("汽车起动电池", "汽车蓄电池", "起停电池", "铅酸汽车电池")),
|
||||
("CHAIN_AF_UP_FLUID", ("车用润滑油", "汽车润滑油", "汽车养护品", "防冻液生产", "制动液生产")),
|
||||
("CHAIN_AF_UP_PART", ("汽车维修配件", "汽车售后配件", "替换零部件", "售后零部件")),
|
||||
("CHAIN_AF_MID_DEALER", ("汽车经销业务", "汽车4s店", "乘用车经销", "汽车销售服务")),
|
||||
("CHAIN_AF_MID_PART", ("汽车配件经销", "汽配供应链", "汽配流通", "汽车零部件分销")),
|
||||
("CHAIN_AF_MID_PLATFORM", ("汽车交易平台", "汽车电商平台", "新车交易平台")),
|
||||
("CHAIN_AF_DOWN_REPAIR", ("汽车维修保养", "汽车维修服务", "汽车美容服务", "汽车售后服务")),
|
||||
("CHAIN_AF_DOWN_TEST", ("机动车检测服务", "汽车检测服务", "车辆检测认证")),
|
||||
("CHAIN_AF_DOWN_USED", ("二手车交易", "二手车经销", "二手车拍卖", "二手车平台")),
|
||||
("CHAIN_AF_DOWN_RECYCLE", ("报废汽车拆解", "汽车拆解业务", "报废机动车回收", "动力电池回收拆解")),
|
||||
("CHAIN_AF_DOWN_RENT", ("汽车租赁业务", "车辆租赁服务", "汽车融资租赁")),
|
||||
("CHAIN_AF_DOWN_ROADSIDE", ("汽车道路救援", "车辆救援服务", "汽车延保服务")),
|
||||
("CHAIN_AF_DOWN_MOBILITY", ("网约车运营", "出租车运营", "汽车出行服务", "共享汽车运营")),
|
||||
)
|
||||
@@ -0,0 +1,222 @@
|
||||
"""Industry definitions and product rules: healthcare."""
|
||||
|
||||
INNOVATIVE_DRUG_DEFINITIONS = (
|
||||
("CHAIN_DRUG", "创新药产业链", "", 1),
|
||||
("CHAIN_DRUG_UP", "上游研发与供应", "CHAIN_DRUG", 2),
|
||||
("CHAIN_DRUG_UP_TOOL", "科研试剂与生命科学工具", "CHAIN_DRUG_UP", 3),
|
||||
("CHAIN_DRUG_UP_RAW", "原料药/辅料/耗材", "CHAIN_DRUG_UP", 3),
|
||||
("CHAIN_DRUG_UP_CRO", "CRO与临床前服务", "CHAIN_DRUG_UP", 3),
|
||||
("CHAIN_DRUG_UP_CDMO", "CDMO/CMO", "CHAIN_DRUG_UP", 3),
|
||||
("CHAIN_DRUG_MID", "药物研发与制造", "CHAIN_DRUG", 2),
|
||||
("CHAIN_DRUG_MID_CHEM", "小分子创新药", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_MID_BIO", "抗体与重组蛋白", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_MID_ADC", "ADC与多抗", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_MID_CELL", "细胞与基因治疗", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_MID_VACCINE", "创新疫苗", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_MID_CLINICAL", "临床开发与注册", "CHAIN_DRUG_MID", 3),
|
||||
("CHAIN_DRUG_DOWN", "商业化与服务", "CHAIN_DRUG", 2),
|
||||
("CHAIN_DRUG_DOWN_MANUFACTURE", "药品生产", "CHAIN_DRUG_DOWN", 3),
|
||||
("CHAIN_DRUG_DOWN_DISTRIBUTION", "医药流通与零售", "CHAIN_DRUG_DOWN", 3),
|
||||
("CHAIN_DRUG_DOWN_SERVICE", "诊疗与支付", "CHAIN_DRUG_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
INNOVATIVE_DRUG_RULES = (
|
||||
("CHAIN_DRUG_UP_TOOL", ("生命科学试剂", "科研试剂", "基因测序试剂", "细胞培养基", "生命科学工具")),
|
||||
("CHAIN_DRUG_UP_RAW", ("创新药原料药", "药用辅料", "制药耗材", "一次性生物工艺耗材")),
|
||||
("CHAIN_DRUG_UP_CRO", ("药物发现cro", "临床前cro", "临床cro", "医药研发外包", "合同研究组织", "临床前安全性评价", "药物安全性评价", "临床前研究服务", "新药研发和生产服务", "药物发现与药学研究", "医药研发服务", "药物研发并提供技术咨询")),
|
||||
("CHAIN_DRUG_UP_CDMO", ("医药cdmo", "生物药cdmo", "小分子cdmo", "细胞与基因治疗cdmo", "基因治疗cdmo", "合同研发生产")),
|
||||
("CHAIN_DRUG_MID_CHEM", ("小分子创新药", "创新化学药", "first-in-class小分子")),
|
||||
("CHAIN_DRUG_MID_BIO", ("创新抗体药", "单克隆抗体药物", "重组蛋白药物", "重组人胰岛素", "胰岛素类似物", "双特异性抗体")),
|
||||
("CHAIN_DRUG_MID_ADC", ("adc药物", "抗体偶联药物", "多特异性抗体", "三抗药物")),
|
||||
("CHAIN_DRUG_MID_CELL", ("car-t", "细胞治疗", "基因治疗", "溶瘤病毒", "核酸药物")),
|
||||
("CHAIN_DRUG_MID_VACCINE", ("创新疫苗", "mrna疫苗", "重组疫苗")),
|
||||
("CHAIN_DRUG_MID_CLINICAL", ("创新药临床试验", "新药注册申报", "临床开发服务")),
|
||||
("CHAIN_DRUG_DOWN_MANUFACTURE", ("创新药生产", "生物药生产", "商业化药品生产")),
|
||||
("CHAIN_DRUG_DOWN_DISTRIBUTION", ("创新药商业化", "创新药销售", "医药流通", "药品零售")),
|
||||
("CHAIN_DRUG_DOWN_SERVICE", ("创新药诊疗", "特药服务", "商保支付", "患者管理服务", "细胞免疫治疗中心", "细胞治疗医疗服务", "肿瘤诊疗服务")),
|
||||
)
|
||||
|
||||
|
||||
MEDICAL_DEVICE_DEFINITIONS = (
|
||||
("CHAIN_MED", "医疗器械产业链", "", 1),
|
||||
("CHAIN_MED_UP", "上游材料与零部件", "CHAIN_MED", 2),
|
||||
("CHAIN_MED_UP_MATERIAL", "医用材料与耗材", "CHAIN_MED_UP", 3),
|
||||
("CHAIN_MED_UP_COMPONENT", "核心零部件与芯片", "CHAIN_MED_UP", 3),
|
||||
("CHAIN_MED_UP_CRO", "器械CRO与注册服务", "CHAIN_MED_UP", 3),
|
||||
("CHAIN_MED_MID", "器械研发制造", "CHAIN_MED", 2),
|
||||
("CHAIN_MED_MID_IVD", "体外诊断", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_IMAGING", "医学影像", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_MONITOR", "监护与生命支持", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_SURGERY", "手术与介入器械", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_ROBOT", "手术机器人", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_IMPLANT", "植介入与骨科器械", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_HOME", "家用与康复器械", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_MID_BCI", "脑机接口", "CHAIN_MED_MID", 3),
|
||||
("CHAIN_MED_DOWN", "流通与医疗服务", "CHAIN_MED", 2),
|
||||
("CHAIN_MED_DOWN_DISTRIBUTION", "器械流通与供应链", "CHAIN_MED_DOWN", 3),
|
||||
("CHAIN_MED_DOWN_HOSPITAL", "医院与第三方服务", "CHAIN_MED_DOWN", 3),
|
||||
("CHAIN_MED_DOWN_MAINTENANCE", "设备运维与数字化", "CHAIN_MED_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
MEDICAL_DEVICE_RULES = (
|
||||
("CHAIN_MED_UP_MATERIAL", ("医用高分子材料", "医用金属材料", "医疗器械耗材", "医用导管材料")),
|
||||
("CHAIN_MED_UP_COMPONENT", ("医疗器械芯片", "医疗传感器", "医疗影像探测器", "医疗设备核心部件")),
|
||||
("CHAIN_MED_UP_CRO", ("医疗器械cro", "器械临床试验", "医疗器械注册服务", "器械研发外包", "医疗器械检测", "器械检测认证")),
|
||||
("CHAIN_MED_MID_IVD", ("体外诊断", "ivd试剂", "分子诊断", "免疫诊断", "生化诊断")),
|
||||
("CHAIN_MED_MID_IMAGING", ("医学影像设备", "ct设备", "磁共振设备", "超声诊断设备", "数字x射线")),
|
||||
("CHAIN_MED_MID_MONITOR", ("医疗监护仪", "呼吸机", "麻醉机", "血液净化设备", "生命支持设备")),
|
||||
("CHAIN_MED_MID_ROBOT", ("手术机器人", "腔镜机器人", "骨科手术机器人")),
|
||||
("CHAIN_MED_MID_SURGERY", ("手术器械", "内窥镜", "介入器械", "微创手术器械")),
|
||||
("CHAIN_MED_MID_IMPLANT", ("骨科植入物", "人工关节", "心血管介入", "植入式医疗器械")),
|
||||
("CHAIN_MED_MID_HOME", ("家用医疗器械", "康复医疗器械", "血糖仪", "制氧机")),
|
||||
("CHAIN_MED_MID_BCI", ("脑机接口", "侵入式脑机", "神经调控设备")),
|
||||
("CHAIN_MED_DOWN_DISTRIBUTION", ("医疗器械流通", "医疗器械供应链", "器械经销")),
|
||||
("CHAIN_MED_DOWN_HOSPITAL", ("第三方医学检验", "医学影像中心", "医院设备服务")),
|
||||
("CHAIN_MED_DOWN_MAINTENANCE", ("医疗设备运维", "医疗设备管理", "医疗器械数字化")),
|
||||
)
|
||||
|
||||
|
||||
CHEMICAL_PHARMA_DEFINITIONS = (
|
||||
("CHAIN_PHARM", "化学制药产业链", "", 1),
|
||||
("CHAIN_PHARM_UP", "上游原辅料", "CHAIN_PHARM", 2),
|
||||
("CHAIN_PHARM_UP_INTERMEDIATE", "医药中间体", "CHAIN_PHARM_UP", 3),
|
||||
("CHAIN_PHARM_UP_API", "大宗原料药", "CHAIN_PHARM_UP", 3),
|
||||
("CHAIN_PHARM_UP_API_SPECIAL", "特色原料药", "CHAIN_PHARM_UP", 3),
|
||||
("CHAIN_PHARM_UP_EXCIPIENT", "药用辅料与包材", "CHAIN_PHARM_UP", 3),
|
||||
("CHAIN_PHARM_MID", "中游制剂制造", "CHAIN_PHARM", 2),
|
||||
("CHAIN_PHARM_MID_GENERIC", "化学仿制药", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_IMPROVED", "改良型新药", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_ORAL", "口服固体制剂", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_INJECTION", "注射剂", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_COMPLEX", "复杂制剂", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_SPECIALTY", "专科化学药", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_MID_GENERAL", "综合药品制剂", "CHAIN_PHARM_MID", 3),
|
||||
("CHAIN_PHARM_DOWN", "下游生产与商业化", "CHAIN_PHARM", 2),
|
||||
("CHAIN_PHARM_DOWN_CMO", "制剂CMO/CDMO", "CHAIN_PHARM_DOWN", 3),
|
||||
("CHAIN_PHARM_DOWN_LICENSE", "药品许可与商业化", "CHAIN_PHARM_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
CHEMICAL_PHARMA_RULES = (
|
||||
("CHAIN_PHARM_UP_INTERMEDIATE", ("医药中间体", "药物中间体", "原料药中间体")),
|
||||
("CHAIN_PHARM_UP_API_SPECIAL", ("特色原料药", "高端原料药", "专利原料药", "小品种原料药")),
|
||||
("CHAIN_PHARM_UP_API", ("大宗原料药", "抗生素原料药", "维生素原料药", "解热镇痛原料药", "化学原料药")),
|
||||
("CHAIN_PHARM_UP_EXCIPIENT", ("药用辅料", "药用胶囊", "药用包材", "药用玻璃", "药用橡胶塞")),
|
||||
("CHAIN_PHARM_MID_IMPROVED", ("改良型新药", "改良新药", "二类新药")),
|
||||
("CHAIN_PHARM_MID_COMPLEX", ("吸入制剂", "缓控释制剂", "脂质体注射剂", "微球制剂", "透皮制剂", "眼用制剂")),
|
||||
("CHAIN_PHARM_MID_INJECTION", ("化学药注射剂", "小容量注射剂", "大容量注射剂", "冻干粉针剂")),
|
||||
("CHAIN_PHARM_MID_ORAL", ("口服固体制剂", "化学药片剂", "化学药胶囊剂", "口服制剂")),
|
||||
("CHAIN_PHARM_MID_GENERIC", ("化学仿制药", "仿制药制剂", "一致性评价品种")),
|
||||
("CHAIN_PHARM_MID_SPECIALTY", ("麻醉药品", "精神类药品", "造影剂", "抗感染化学药", "心脑血管化学药", "抗肿瘤化学药")),
|
||||
("CHAIN_PHARM_MID_GENERAL", ("药品的研发、生产和销售", "药品研发、生产、销售", "药品研发、生产、销售业务", "新药研发、生产制造、推广营销业务", "医药产品的研发、生产和销售")),
|
||||
("CHAIN_PHARM_DOWN_CMO", ("化学制剂cmo", "制剂cdmo", "药品委托生产", "制剂受托生产")),
|
||||
("CHAIN_PHARM_DOWN_LICENSE", ("仿制药商业化", "药品许可引进", "化学药销售推广")),
|
||||
)
|
||||
|
||||
|
||||
TCM_DEFINITIONS = (
|
||||
("CHAIN_TCM", "中药产业链", "", 1),
|
||||
("CHAIN_TCM_UP", "上游中药资源", "CHAIN_TCM", 2),
|
||||
("CHAIN_TCM_UP_PLANT", "中药材种植与养殖", "CHAIN_TCM_UP", 3),
|
||||
("CHAIN_TCM_UP_MARKET", "中药材交易与供应", "CHAIN_TCM_UP", 3),
|
||||
("CHAIN_TCM_UP_EXTRACT", "中药提取物", "CHAIN_TCM_UP", 3),
|
||||
("CHAIN_TCM_MID", "中游加工制造", "CHAIN_TCM", 2),
|
||||
("CHAIN_TCM_MID_PIECES", "中药饮片", "CHAIN_TCM_MID", 3),
|
||||
("CHAIN_TCM_MID_GRANULE", "中药配方颗粒", "CHAIN_TCM_MID", 3),
|
||||
("CHAIN_TCM_MID_PATENT", "中成药", "CHAIN_TCM_MID", 3),
|
||||
("CHAIN_TCM_MID_CLASSIC", "经典名方与院内制剂", "CHAIN_TCM_MID", 3),
|
||||
("CHAIN_TCM_MID_ETHNIC", "民族药", "CHAIN_TCM_MID", 3),
|
||||
("CHAIN_TCM_DOWN", "下游健康产品与服务", "CHAIN_TCM", 2),
|
||||
("CHAIN_TCM_DOWN_HEALTH", "药食同源与中药保健品", "CHAIN_TCM_DOWN", 3),
|
||||
("CHAIN_TCM_DOWN_CLINIC", "中医诊疗服务", "CHAIN_TCM_DOWN", 3),
|
||||
("CHAIN_TCM_DOWN_COMMERCE", "中药流通与零售", "CHAIN_TCM_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
TCM_RULES = (
|
||||
("CHAIN_TCM_UP_PLANT", ("中药材种植", "道地药材种植", "中药材养殖", "中药材基地")),
|
||||
("CHAIN_TCM_UP_MARKET", ("中药材交易", "中药材供应", "中药材仓储", "中药材市场")),
|
||||
("CHAIN_TCM_UP_EXTRACT", ("中药提取物", "植物提取物", "天然药物提取")),
|
||||
("CHAIN_TCM_MID_PIECES", ("中药饮片", "精制饮片", "毒性饮片")),
|
||||
("CHAIN_TCM_MID_GRANULE", ("中药配方颗粒", "单味配方颗粒")),
|
||||
("CHAIN_TCM_MID_CLASSIC", ("经典名方", "中药院内制剂", "古代经典名方")),
|
||||
("CHAIN_TCM_MID_ETHNIC", ("藏药", "蒙药", "维药", "民族药")),
|
||||
("CHAIN_TCM_MID_PATENT", ("中成药", "中药制剂", "中药口服液", "中药注射剂", "中药贴膏")),
|
||||
("CHAIN_TCM_DOWN_HEALTH", ("药食同源产品", "中药保健食品", "中药大健康产品", "中药养生产品")),
|
||||
("CHAIN_TCM_DOWN_CLINIC", ("中医诊疗", "中医医院", "中医门诊", "中医医疗服务")),
|
||||
("CHAIN_TCM_DOWN_COMMERCE", ("中药商业", "中药流通", "中药零售", "中药材电商")),
|
||||
)
|
||||
|
||||
|
||||
MEDICAL_SERVICE_DEFINITIONS = (
|
||||
("CHAIN_CARE", "医疗服务产业链", "", 1),
|
||||
("CHAIN_CARE_UP", "上游运营支持", "CHAIN_CARE", 2),
|
||||
("CHAIN_CARE_UP_SPD", "医疗供应链SPD", "CHAIN_CARE_UP", 3),
|
||||
("CHAIN_CARE_UP_RESEARCH", "临床研究与转化服务", "CHAIN_CARE_UP", 3),
|
||||
("CHAIN_CARE_UP_DIGITAL", "医院数字化运营", "CHAIN_CARE_UP", 3),
|
||||
("CHAIN_CARE_MID", "中游诊断与医疗机构", "CHAIN_CARE", 2),
|
||||
("CHAIN_CARE_MID_GENERAL", "综合医院", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_EYE", "眼科医疗", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_DENTAL", "口腔医疗", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_ONCOLOGY", "肿瘤医疗", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_REPRO", "辅助生殖", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_OTHER", "其他专科医疗", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_ICL", "第三方医学检验", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_MID_CHECKUP", "体检与健康管理", "CHAIN_CARE_MID", 3),
|
||||
("CHAIN_CARE_DOWN", "下游康复与延伸服务", "CHAIN_CARE", 2),
|
||||
("CHAIN_CARE_DOWN_REHAB", "康复医疗", "CHAIN_CARE_DOWN", 3),
|
||||
("CHAIN_CARE_DOWN_ELDER", "养老与护理", "CHAIN_CARE_DOWN", 3),
|
||||
("CHAIN_CARE_DOWN_INTERNET", "互联网医疗服务", "CHAIN_CARE_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
MEDICAL_SERVICE_RULES = (
|
||||
("CHAIN_CARE_UP_SPD", ("医疗spd", "医用耗材spd", "医疗供应链精细化管理", "医院供应链管理服务")),
|
||||
("CHAIN_CARE_UP_RESEARCH", ("医院临床研究服务", "医学成果转化", "临床试验机构服务")),
|
||||
("CHAIN_CARE_UP_DIGITAL", ("医院运营数字化", "智慧医院运营", "医院信息化服务")),
|
||||
("CHAIN_CARE_MID_GENERAL", ("综合医院运营", "综合性医院", "综合医疗服务")),
|
||||
("CHAIN_CARE_MID_EYE", ("眼科医院", "眼科医疗服务", "眼科诊疗")),
|
||||
("CHAIN_CARE_MID_DENTAL", ("口腔医院", "口腔医疗服务", "口腔诊疗")),
|
||||
("CHAIN_CARE_MID_ONCOLOGY", ("肿瘤医院", "肿瘤医疗服务", "肿瘤诊疗中心")),
|
||||
("CHAIN_CARE_MID_REPRO", ("辅助生殖医疗", "生殖医院", "生殖医学中心")),
|
||||
("CHAIN_CARE_MID_ICL", ("第三方医学检验", "独立医学实验室", "医学检验服务")),
|
||||
("CHAIN_CARE_MID_CHECKUP", ("健康体检", "体检中心", "健康管理服务")),
|
||||
("CHAIN_CARE_MID_OTHER", ("专科医院", "脑科医院", "骨科医院", "妇儿医院", "精神专科医院")),
|
||||
("CHAIN_CARE_DOWN_REHAB", ("康复医院", "康复医疗服务", "康复诊疗")),
|
||||
("CHAIN_CARE_DOWN_ELDER", ("养老医疗", "医养结合", "护理院", "养老护理服务")),
|
||||
("CHAIN_CARE_DOWN_INTERNET", ("互联网医院", "在线诊疗服务", "互联网医疗服务")),
|
||||
)
|
||||
|
||||
|
||||
PHARMA_DISTRIBUTION_DEFINITIONS = (
|
||||
("CHAIN_DIST", "医药流通产业链", "", 1),
|
||||
("CHAIN_DIST_UP", "上游物流与数字化", "CHAIN_DIST", 2),
|
||||
("CHAIN_DIST_UP_LOGISTICS", "医药仓储物流", "CHAIN_DIST_UP", 3),
|
||||
("CHAIN_DIST_UP_COLD", "医药冷链", "CHAIN_DIST_UP", 3),
|
||||
("CHAIN_DIST_UP_IT", "流通数字化与追溯", "CHAIN_DIST_UP", 3),
|
||||
("CHAIN_DIST_MID", "中游批发与供应链", "CHAIN_DIST", 2),
|
||||
("CHAIN_DIST_MID_WHOLESALE", "医药批发配送", "CHAIN_DIST_MID", 3),
|
||||
("CHAIN_DIST_MID_DEVICE", "器械与耗材配送", "CHAIN_DIST_MID", 3),
|
||||
("CHAIN_DIST_MID_SPD", "医院SPD与院内物流", "CHAIN_DIST_MID", 3),
|
||||
("CHAIN_DIST_MID_DTP", "DTP与特药服务", "CHAIN_DIST_MID", 3),
|
||||
("CHAIN_DIST_DOWN", "下游零售与服务", "CHAIN_DIST", 2),
|
||||
("CHAIN_DIST_DOWN_CHAIN", "连锁药店", "CHAIN_DIST_DOWN", 3),
|
||||
("CHAIN_DIST_DOWN_ECOM", "医药电商", "CHAIN_DIST_DOWN", 3),
|
||||
("CHAIN_DIST_DOWN_SERVICE", "慢病与处方服务", "CHAIN_DIST_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
PHARMA_DISTRIBUTION_RULES = (
|
||||
("CHAIN_DIST_UP_COLD", ("医药冷链物流", "药品冷链", "疫苗冷链", "生物医药冷链")),
|
||||
("CHAIN_DIST_UP_LOGISTICS", ("医药仓储物流", "药品物流中心", "医药物流配送")),
|
||||
("CHAIN_DIST_UP_IT", ("药品追溯系统", "医药流通数字化", "药品供应链信息系统")),
|
||||
("CHAIN_DIST_MID_WHOLESALE", ("医药批发", "药品批发配送", "医药商业分销", "药品供应链服务")),
|
||||
("CHAIN_DIST_MID_DEVICE", ("医疗器械配送", "医用耗材配送", "器械商业分销")),
|
||||
("CHAIN_DIST_MID_SPD", ("医院spd", "院内物流服务", "医用耗材供应链管理")),
|
||||
("CHAIN_DIST_MID_DTP", ("dtp药房", "特药配送", "特药服务", "处方外流承接")),
|
||||
("CHAIN_DIST_DOWN_CHAIN", ("连锁药店", "医药零售门店", "零售药房")),
|
||||
("CHAIN_DIST_DOWN_ECOM", ("医药电商", "网上药店", "药品电商平台")),
|
||||
("CHAIN_DIST_DOWN_SERVICE", ("慢病管理服务", "处方药零售服务", "患者管理服务")),
|
||||
)
|
||||
@@ -0,0 +1,459 @@
|
||||
"""Industry definitions and product rules: services infrastructure."""
|
||||
|
||||
INFRASTRUCTURE_DEFINITIONS = (
|
||||
("CHAIN_INFRA", "基础设施工程产业链", "", 1),
|
||||
("CHAIN_INFRA_UP", "上游材料与装备", "CHAIN_INFRA", 2),
|
||||
("CHAIN_INFRA_UP_MATERIAL", "建筑材料", "CHAIN_INFRA_UP", 3),
|
||||
("CHAIN_INFRA_UP_EQUIPMENT", "工程机械与设备", "CHAIN_INFRA_UP", 3),
|
||||
("CHAIN_INFRA_UP_EQ_EARTH", "土方机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_LIFT", "起重机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_CONCRETE", "混凝土机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_ROAD", "道路机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_TUNNEL", "桩工与隧道机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_MINING", "矿山机械", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_AERIAL", "高空作业与工业车辆", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_PART", "液压件与核心零部件", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_UP_EQ_SERVICE", "租赁与维保服务", "CHAIN_INFRA_UP_EQUIPMENT", 4),
|
||||
("CHAIN_INFRA_MID", "中游工程建设", "CHAIN_INFRA", 2),
|
||||
("CHAIN_INFRA_MID_SURVEY", "勘察设计与监测", "CHAIN_INFRA_MID", 3),
|
||||
("CHAIN_INFRA_MID_GEOTECH", "岩土与地基工程", "CHAIN_INFRA_MID", 3),
|
||||
("CHAIN_INFRA_MID_CIVIL", "土木与建筑施工", "CHAIN_INFRA_MID", 3),
|
||||
("CHAIN_INFRA_MID_CONTRACT", "工程总承包", "CHAIN_INFRA_MID", 3),
|
||||
("CHAIN_INFRA_DOWN", "下游基础设施", "CHAIN_INFRA", 2),
|
||||
("CHAIN_INFRA_DOWN_TRANSPORT", "交通基础设施", "CHAIN_INFRA_DOWN", 3),
|
||||
("CHAIN_INFRA_DOWN_MUNICIPAL", "市政与水利", "CHAIN_INFRA_DOWN", 3),
|
||||
("CHAIN_INFRA_DOWN_PORT", "港口航道", "CHAIN_INFRA_DOWN", 3),
|
||||
("CHAIN_INFRA_DOWN_ENERGY", "新能源基础设施", "CHAIN_INFRA_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
INFRASTRUCTURE_RULES = (
|
||||
("CHAIN_INFRA_UP_EQ_EARTH", ("挖掘机整机", "装载机整机", "推土机整机", "平地机整机", "土方机械")),
|
||||
("CHAIN_INFRA_UP_EQ_LIFT", ("汽车起重机", "履带起重机", "塔式起重机", "起重机械整机")),
|
||||
("CHAIN_INFRA_UP_EQ_CONCRETE", ("混凝土泵车", "混凝土搅拌车", "混凝土搅拌站", "混凝土机械")),
|
||||
("CHAIN_INFRA_UP_EQ_ROAD", ("压路机整机", "摊铺机整机", "路面机械", "道路养护机械")),
|
||||
("CHAIN_INFRA_UP_EQ_TUNNEL", ("盾构机", "隧道掘进机", "旋挖钻机", "桩工机械")),
|
||||
("CHAIN_INFRA_UP_EQ_MINING", ("矿山机械", "矿用挖掘机", "破碎筛分设备", "矿山成套设备")),
|
||||
("CHAIN_INFRA_UP_EQ_AERIAL", ("高空作业平台", "叉车整机", "工业车辆", "升降工作平台")),
|
||||
("CHAIN_INFRA_UP_EQ_PART", ("工程机械液压件", "液压油缸", "液压泵阀", "工程机械传动件", "工程机械底盘件")),
|
||||
("CHAIN_INFRA_UP_EQ_SERVICE", ("工程机械租赁", "工程机械维修", "工程机械再制造", "工程机械后市场")),
|
||||
("CHAIN_INFRA_MID_GEOTECH", ("岩土工程", "地基处理", "桩基工程", "软基础处理", "地基与基础工程", "基坑工程")),
|
||||
("CHAIN_INFRA_MID_SURVEY", ("工程勘察设计", "岩土勘察", "工程监测", "工程检测", "测绘工程")),
|
||||
("CHAIN_INFRA_MID_CONTRACT", ("工程总承包", "施工总承包", "epc总承包", "工程承包", "数据中心工程总承包", "智算中心工程总承包")),
|
||||
("CHAIN_INFRA_MID_CIVIL", ("土木工程施工", "建筑工程施工", "基础设施建设", "工程建设施工")),
|
||||
("CHAIN_INFRA_UP_MATERIAL", ("建筑材料", "水泥制品", "预应力混凝土", "工程防水材料", "土工材料")),
|
||||
("CHAIN_INFRA_UP_EQUIPMENT", ("工程机械", "施工设备", "桩工机械", "盾构机", "挖掘机械")),
|
||||
("CHAIN_INFRA_DOWN_PORT", ("港口工程", "航道工程", "河道疏浚", "码头工程")),
|
||||
("CHAIN_INFRA_DOWN_MUNICIPAL", ("市政公用工程", "水利工程", "地下管网", "城市更新")),
|
||||
("CHAIN_INFRA_DOWN_TRANSPORT", ("公路工程", "铁路工程", "轨道交通工程", "桥梁工程")),
|
||||
("CHAIN_INFRA_DOWN_ENERGY", ("新能源基建", "新能源基础设施", "光伏电站建设", "风电场建设")),
|
||||
)
|
||||
|
||||
|
||||
EDUCATION_DEFINITIONS = (
|
||||
("CHAIN_EDU", "教育产业链", "", 1),
|
||||
("CHAIN_EDU_UP", "上游内容与技术", "CHAIN_EDU", 2),
|
||||
("CHAIN_EDU_UP_CONTENT", "教材与课程内容", "CHAIN_EDU_UP", 3),
|
||||
("CHAIN_EDU_UP_TECH", "教育信息化与AI教育", "CHAIN_EDU_UP", 3),
|
||||
("CHAIN_EDU_UP_EQUIPMENT", "教学设备", "CHAIN_EDU_UP", 3),
|
||||
("CHAIN_EDU_MID", "中游教育服务", "CHAIN_EDU", 2),
|
||||
("CHAIN_EDU_MID_GENERAL", "综合教育培训", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_VOCATIONAL", "职业教育", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_HIGHER", "高等教育", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_EXAM", "招录与资格考试培训", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_LANGUAGE", "语言与留学服务", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_K12", "素质教育与课后服务", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_MID_DRIVING", "驾驶培训", "CHAIN_EDU_MID", 3),
|
||||
("CHAIN_EDU_DOWN", "下游平台与运营", "CHAIN_EDU", 2),
|
||||
("CHAIN_EDU_DOWN_ONLINE", "在线教育平台", "CHAIN_EDU_DOWN", 3),
|
||||
("CHAIN_EDU_DOWN_SCHOOL", "学校运营与后勤", "CHAIN_EDU_DOWN", 3),
|
||||
("CHAIN_EDU_DOWN_ENTERPRISE", "企业培训", "CHAIN_EDU_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
EDUCATION_RULES = (
|
||||
("CHAIN_EDU_UP_CONTENT", ("教育教材", "数字教材", "教材出版", "教育出版", "课程内容研发", "课程研发", "教育内容出版")),
|
||||
("CHAIN_EDU_UP_TECH", ("教育信息化业务", "教育信息化产品", "教育信息化解决方案", "教育教学信息化解决方案", "教育信息化", "智慧教育", "智慧教育平台", "建设了智慧教育云平台", "智慧教育业务", "教育数字化产品", "教育数字化解决方案", "智慧校园应用解决方案", "智慧校园云平台", "移动教育业务", "ai教育产品", "ai教育平台", "人工智能教学", "教学管理软件", "教育软件")),
|
||||
("CHAIN_EDU_UP_EQUIPMENT", ("教学设备", "智慧教室设备", "教育录播设备", "教育交互平板", "教学交互平板", "iwb产品", "智慧黑板", "教育智能硬件")),
|
||||
("CHAIN_EDU_MID_VOCATIONAL", ("数字化人才培养", "非学历职业教育", "职业教育业务", "职业教育服务", "职业教育培训", "职业培训业务", "职业技能培训服务", "职业技能培训机构", "职业学校运营")),
|
||||
("CHAIN_EDU_MID_HIGHER", ("高等教育服务", "民办大学", "高校运营")),
|
||||
("CHAIN_EDU_MID_EXAM", ("公务员考试培训", "公务员招录培训", "公职考试培训", "事业单位招录培训", "招录考试培训", "职业资格考试培训")),
|
||||
("CHAIN_EDU_MID_LANGUAGE", ("语言培训", "留学咨询", "留学服务", "国际教育服务")),
|
||||
("CHAIN_EDU_MID_K12", ("个性化教育", "全日制教育", "k12教育", "中小学教育", "素质教育", "课后服务", "非学科培训", "教育托管服务")),
|
||||
("CHAIN_EDU_MID_DRIVING", ("机动车驾驶培训", "驾校培训", "驾驶员培训")),
|
||||
("CHAIN_EDU_MID_GENERAL", ("教育培训业务", "教育培训服务", "非学历培训业务", "短期培训", "学历教育", "非学历教育")),
|
||||
("CHAIN_EDU_DOWN_ONLINE", ("运营在线教育平台", "在线教育平台运营", "旗下在线教育平台", "在线教育平台服务", "互联网教育平台运营", "在线课程平台运营")),
|
||||
("CHAIN_EDU_DOWN_SCHOOL", ("学校运营", "校园后勤服务", "教育园区运营")),
|
||||
("CHAIN_EDU_DOWN_ENTERPRISE", ("企业培训服务", "企业学习平台", "管理培训服务")),
|
||||
)
|
||||
|
||||
|
||||
HR_SERVICE_DEFINITIONS = (
|
||||
("CHAIN_HR", "人力资源服务产业链", "", 1),
|
||||
("CHAIN_HR_UP", "上游平台与技术", "CHAIN_HR", 2),
|
||||
("CHAIN_HR_UP_RECRUIT", "在线招聘平台", "CHAIN_HR_UP", 3),
|
||||
("CHAIN_HR_UP_DIGITAL", "人力资源数字化", "CHAIN_HR_UP", 3),
|
||||
("CHAIN_HR_MID", "中游用工服务", "CHAIN_HR", 2),
|
||||
("CHAIN_HR_MID_COMPREHENSIVE", "综合人力资源服务", "CHAIN_HR_MID", 3),
|
||||
("CHAIN_HR_MID_STAFFING", "人才派遣与招聘", "CHAIN_HR_MID", 3),
|
||||
("CHAIN_HR_MID_FLEX", "灵活用工", "CHAIN_HR_MID", 3),
|
||||
("CHAIN_HR_MID_OUTSOURCE", "业务流程外包", "CHAIN_HR_MID", 3),
|
||||
("CHAIN_HR_MID_RPO", "猎头与RPO", "CHAIN_HR_MID", 3),
|
||||
("CHAIN_HR_DOWN", "下游人才与管理服务", "CHAIN_HR", 2),
|
||||
("CHAIN_HR_DOWN_TRAIN", "人才培训", "CHAIN_HR_DOWN", 3),
|
||||
("CHAIN_HR_DOWN_PAYROLL", "薪税与福利服务", "CHAIN_HR_DOWN", 3),
|
||||
("CHAIN_HR_DOWN_CONSULT", "人力资源咨询", "CHAIN_HR_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
HR_SERVICE_RULES = (
|
||||
("CHAIN_HR_UP_RECRUIT", ("在线招聘平台", "互联网招聘平台", "招聘网站运营")),
|
||||
("CHAIN_HR_UP_DIGITAL", ("人力资源saas", "人力资源管理系统", "数字化人力资源平台", "人力资源产业互联平台", "组织人事管理产品", "干部人事管理产品")),
|
||||
("CHAIN_HR_MID_STAFFING", ("人才派遣服务", "劳务派遣服务", "人力资源招聘服务", "招聘服务")),
|
||||
("CHAIN_HR_MID_FLEX", ("灵活用工服务", "灵活用工平台", "弹性用工服务")),
|
||||
("CHAIN_HR_MID_OUTSOURCE", ("人力资源业务外包", "人力资源流程外包", "岗位外包服务")),
|
||||
("CHAIN_HR_MID_RPO", ("招聘流程外包", "rpo服务", "猎头服务")),
|
||||
("CHAIN_HR_DOWN_TRAIN", ("人才培训服务", "职业人才培训", "员工培训服务")),
|
||||
("CHAIN_HR_DOWN_PAYROLL", ("薪酬服务", "薪税服务", "员工福利服务", "人事代理服务")),
|
||||
("CHAIN_HR_DOWN_CONSULT", ("人力资源咨询", "人才管理咨询", "组织管理咨询")),
|
||||
("CHAIN_HR_MID_COMPREHENSIVE", ("综合人力资源服务", "人力资源服务整体解决方案")),
|
||||
)
|
||||
|
||||
|
||||
MEDIA_CONTENT_DEFINITIONS = (
|
||||
("CHAIN_MEDIA", "传媒内容产业链", "", 1),
|
||||
("CHAIN_MEDIA_UP", "上游内容资产与技术", "CHAIN_MEDIA", 2),
|
||||
("CHAIN_MEDIA_UP_IP", "IP与版权资产", "CHAIN_MEDIA_UP", 3),
|
||||
("CHAIN_MEDIA_UP_AIGC", "AIGC与数字内容技术", "CHAIN_MEDIA_UP", 3),
|
||||
("CHAIN_MEDIA_UP_PRODUCTION", "制作与后期技术", "CHAIN_MEDIA_UP", 3),
|
||||
("CHAIN_MEDIA_MID", "中游内容生产", "CHAIN_MEDIA", 2),
|
||||
("CHAIN_MEDIA_MID_PUBLISH", "图书与数字出版", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_FILM", "电影制作发行", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_TV", "电视剧与综艺制作", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_ANIMATION", "动漫内容", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_SHORT", "短剧与漫剧", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_GAME", "游戏内容", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_MID_GAME_FULL", "客户端与移动游戏", "CHAIN_MEDIA_MID_GAME", 4),
|
||||
("CHAIN_MEDIA_MID_GAME_MINI", "小游戏", "CHAIN_MEDIA_MID_GAME", 4),
|
||||
("CHAIN_MEDIA_MID_AD", "广告创意与内容", "CHAIN_MEDIA_MID", 3),
|
||||
("CHAIN_MEDIA_DOWN", "下游发行运营与商业化", "CHAIN_MEDIA", 2),
|
||||
("CHAIN_MEDIA_DOWN_CINEMA", "院线与影院", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_BROADCAST", "广播电视网络", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_VIDEO", "长视频与流媒体平台", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_SHORT", "短视频与MCN", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_GAME", "游戏发行与运营", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_MARKETING", "广告代理与数字营销", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_DERIVATIVE", "IP授权与衍生消费", "CHAIN_MEDIA_DOWN", 3),
|
||||
("CHAIN_MEDIA_DOWN_OVERSEAS", "内容海外发行", "CHAIN_MEDIA_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
MEDIA_CONTENT_RULES = (
|
||||
("CHAIN_MEDIA_UP_IP", ("数字内容版权交易平台", "视觉内容与服务", "ip版权运营", "内容版权运营", "版权代理", "版权管理", "版权交易")),
|
||||
("CHAIN_MEDIA_UP_AIGC", ("ai视频生成", "aigc内容生产", "ai数字内容", "人工智能内容生成", "数字内容技术")),
|
||||
("CHAIN_MEDIA_UP_PRODUCTION", ("虚拟拍摄", "影视后期制作", "影视特效制作", "数字内容制作技术")),
|
||||
("CHAIN_MEDIA_MID_PUBLISH", ("数字阅读产品", "数字出版运营服务", "网络文学", "图书出版", "教材出版", "期刊出版", "数字出版", "出版发行业务", "出版物编辑")),
|
||||
("CHAIN_MEDIA_MID_FILM", ("电影发行业务", "电影剧集制片", "电影制作", "电影投资制作", "电影发行", "电影制片")),
|
||||
("CHAIN_MEDIA_MID_TV", ("影视项目的投资、制作、发行", "视频内容的制作、发行", "节目销售及制作服务", "电视剧制作", "影视剧制作", "影视内容制作", "综艺节目制作", "电视节目制作")),
|
||||
("CHAIN_MEDIA_MID_ANIMATION", ("动漫影视片", "ip内容业务", "动画制作", "动漫内容", "动漫影视", "动画电影")),
|
||||
("CHAIN_MEDIA_MID_SHORT", ("短剧及ip衍生品", "ai漫剧", "微短剧制作", "短剧制作", "短剧业务", "漫剧制作", "互动短剧")),
|
||||
("CHAIN_MEDIA_MID_GAME_MINI", ("小游戏研发", "微信小游戏", "小程序游戏", "休闲小游戏")),
|
||||
("CHAIN_MEDIA_MID_GAME_FULL", ("网络游戏的研发、发行及运营", "网络游戏的研发及运营", "互联网游戏的研发和运营", "网络游戏的开发及运营", "移动游戏和pc端游戏的研发、发行及运营", "游戏研发", "移动游戏研发", "网络游戏研发", "客户端游戏", "手机游戏", "主机游戏")),
|
||||
("CHAIN_MEDIA_MID_AD", ("广告创意", "广告内容制作", "品牌内容创意")),
|
||||
("CHAIN_MEDIA_DOWN_CINEMA", ("电影院线业务", "影院管理业务", "影院运营", "电影院线", "电影放映业务", "影城运营")),
|
||||
("CHAIN_MEDIA_DOWN_BROADCAST", ("广播电视节目播出", "有线电视网络", "广播电视网络", "电视台运营")),
|
||||
("CHAIN_MEDIA_DOWN_VIDEO", ("长视频平台", "在线视频平台", "视频平台运营", "互联网视频服务")),
|
||||
("CHAIN_MEDIA_DOWN_SHORT", ("mcn运营", "短视频平台", "mcn业务", "达人经纪", "直播经纪")),
|
||||
("CHAIN_MEDIA_DOWN_GAME", ("游戏运营与发行", "游戏运营业务", "游戏发行运营", "网络游戏运营", "移动游戏运营")),
|
||||
("CHAIN_MEDIA_DOWN_MARKETING", ("一站式全营销解决方案", "全案广告代理", "互联网广告及技术服务", "广告代理", "数字营销", "整合营销", "效果营销", "媒介代理", "营销传播服务")),
|
||||
("CHAIN_MEDIA_DOWN_DERIVATIVE", ("ip内容业务", "潮玩业务", "ip授权运营", "ip衍生品", "ip授权业务", "潮玩产品", "动漫衍生品")),
|
||||
("CHAIN_MEDIA_DOWN_OVERSEAS", ("游戏海外发行", "游戏出海", "内容出海", "影视海外发行", "海外内容发行")),
|
||||
)
|
||||
|
||||
|
||||
TRANSPORT_LOGISTICS_DEFINITIONS = (
|
||||
("CHAIN_TRANS", "交通物流产业链", "", 1),
|
||||
("CHAIN_TRANS_UP", "上游基础设施与技术", "CHAIN_TRANS", 2),
|
||||
("CHAIN_TRANS_UP_WAREHOUSE", "仓储与物流园", "CHAIN_TRANS_UP", 3),
|
||||
("CHAIN_TRANS_UP_COLD", "冷链设施", "CHAIN_TRANS_UP", 3),
|
||||
("CHAIN_TRANS_UP_TECH", "物流信息化与装备", "CHAIN_TRANS_UP", 3),
|
||||
("CHAIN_TRANS_MID", "中游运输与节点运营", "CHAIN_TRANS", 2),
|
||||
("CHAIN_TRANS_MID_SHIP", "水运", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_SHIP_CONTAINER", "集装箱航运", "CHAIN_TRANS_MID_SHIP", 4),
|
||||
("CHAIN_TRANS_MID_SHIP_BULK", "干散货航运", "CHAIN_TRANS_MID_SHIP", 4),
|
||||
("CHAIN_TRANS_MID_SHIP_TANKER", "油气品航运", "CHAIN_TRANS_MID_SHIP", 4),
|
||||
("CHAIN_TRANS_MID_PORT", "港口码头运营", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_AIR", "航空运输", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_AIR_PASSENGER", "航空客运", "CHAIN_TRANS_MID_AIR", 4),
|
||||
("CHAIN_TRANS_MID_AIR_CARGO", "航空货运", "CHAIN_TRANS_MID_AIR", 4),
|
||||
("CHAIN_TRANS_MID_AIRPORT", "机场运营", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_RAIL", "铁路运输", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_ROAD", "公路与客运", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_MID_EXPRESS", "快递网络", "CHAIN_TRANS_MID", 3),
|
||||
("CHAIN_TRANS_DOWN", "下游综合物流服务", "CHAIN_TRANS", 2),
|
||||
("CHAIN_TRANS_DOWN_SUPPLY", "供应链与综合物流", "CHAIN_TRANS_DOWN", 3),
|
||||
("CHAIN_TRANS_DOWN_FORWARD", "货运代理", "CHAIN_TRANS_DOWN", 3),
|
||||
("CHAIN_TRANS_DOWN_CROSS", "跨境物流", "CHAIN_TRANS_DOWN", 3),
|
||||
("CHAIN_TRANS_DOWN_LAST", "同城与末端配送", "CHAIN_TRANS_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
TRANSPORT_LOGISTICS_RULES = (
|
||||
("CHAIN_TRANS_UP_WAREHOUSE", ("物流园区运营", "仓储设施运营", "仓储服务", "物流地产")),
|
||||
("CHAIN_TRANS_UP_COLD", ("冷链仓储", "冷链物流", "冷库运营", "医药冷链")),
|
||||
("CHAIN_TRANS_UP_TECH", ("物流信息系统", "智慧物流系统", "物流自动化设备", "物流装备")),
|
||||
("CHAIN_TRANS_MID_SHIP_CONTAINER", ("集装箱航运", "集装箱运输", "班轮运输")),
|
||||
("CHAIN_TRANS_MID_SHIP_BULK", ("干散货航运", "干散货运输", "散货船运输")),
|
||||
("CHAIN_TRANS_MID_SHIP_TANKER", ("油轮运输", "原油运输", "成品油运输", "液化气船运输", "lng运输")),
|
||||
("CHAIN_TRANS_MID_PORT", ("港口装卸", "港口运营", "码头运营", "港口综合服务")),
|
||||
("CHAIN_TRANS_MID_AIR_PASSENGER", ("航空客运", "旅客运输", "国内国际航空客运")),
|
||||
("CHAIN_TRANS_MID_AIR_CARGO", ("航空货运", "航空物流", "航空货邮运输")),
|
||||
("CHAIN_TRANS_MID_AIRPORT", ("机场运营", "机场管理", "航空地面服务")),
|
||||
("CHAIN_TRANS_MID_RAIL", ("铁路运输", "铁路货运", "铁路客运")),
|
||||
("CHAIN_TRANS_MID_ROAD", ("高速公路运营", "收费公路运营", "道路客运", "公交运营")),
|
||||
("CHAIN_TRANS_MID_EXPRESS", ("快递服务", "快递业务", "快递网络")),
|
||||
("CHAIN_TRANS_DOWN_SUPPLY", ("供应链管理服务", "综合物流服务", "一体化物流", "合同物流")),
|
||||
("CHAIN_TRANS_DOWN_FORWARD", ("国际货运代理", "货运代理服务", "海运代理", "航空货代")),
|
||||
("CHAIN_TRANS_DOWN_CROSS", ("跨境物流", "国际物流服务", "跨境电商物流")),
|
||||
("CHAIN_TRANS_DOWN_LAST", ("同城配送", "即时配送", "城市配送", "末端配送")),
|
||||
)
|
||||
|
||||
|
||||
UTILITY_DEFINITIONS = (
|
||||
("CHAIN_UTIL", "公用事业产业链", "", 1),
|
||||
("CHAIN_UTIL_UP", "上游能源获取", "CHAIN_UTIL", 2),
|
||||
("CHAIN_UTIL_UP_FUEL", "燃料与气源采购", "CHAIN_UTIL_UP", 3),
|
||||
("CHAIN_UTIL_UP_LNG", "LNG接收与储运", "CHAIN_UTIL_UP", 3),
|
||||
("CHAIN_UTIL_MID", "中游生产与管网", "CHAIN_UTIL", 2),
|
||||
("CHAIN_UTIL_MID_POWER", "电力生产", "CHAIN_UTIL_MID", 3),
|
||||
("CHAIN_UTIL_MID_POWER_THERMAL", "火力发电", "CHAIN_UTIL_MID_POWER", 4),
|
||||
("CHAIN_UTIL_MID_POWER_HYDRO", "水力发电", "CHAIN_UTIL_MID_POWER", 4),
|
||||
("CHAIN_UTIL_MID_POWER_NUCLEAR", "核电运营", "CHAIN_UTIL_MID_POWER", 4),
|
||||
("CHAIN_UTIL_MID_POWER_RENEW", "新能源发电", "CHAIN_UTIL_MID_POWER", 4),
|
||||
("CHAIN_UTIL_MID_CHP", "热电联产与供热", "CHAIN_UTIL_MID", 3),
|
||||
("CHAIN_UTIL_MID_GRID", "配电网与电网运营", "CHAIN_UTIL_MID", 3),
|
||||
("CHAIN_UTIL_MID_GAS", "燃气输配", "CHAIN_UTIL_MID", 3),
|
||||
("CHAIN_UTIL_MID_GAS_PIPE", "长输管网", "CHAIN_UTIL_MID_GAS", 4),
|
||||
("CHAIN_UTIL_MID_GAS_CITY", "城市燃气", "CHAIN_UTIL_MID_GAS", 4),
|
||||
("CHAIN_UTIL_DOWN", "下游能源服务", "CHAIN_UTIL", 2),
|
||||
("CHAIN_UTIL_DOWN_RETAIL", "售电与综合能源", "CHAIN_UTIL_DOWN", 3),
|
||||
("CHAIN_UTIL_DOWN_GAS", "燃气销售与服务", "CHAIN_UTIL_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
UTILITY_RULES = (
|
||||
("CHAIN_UTIL_UP_FUEL", ("煤炭采购与供应", "天然气气源采购", "燃料采购")),
|
||||
("CHAIN_UTIL_UP_LNG", ("lng接收站运营", "液化天然气接收站", "lng储运")),
|
||||
("CHAIN_UTIL_MID_POWER_THERMAL", ("火力发电", "燃煤发电", "燃气发电")),
|
||||
("CHAIN_UTIL_MID_POWER_HYDRO", ("水力发电", "水电站运营")),
|
||||
("CHAIN_UTIL_MID_POWER_NUCLEAR", ("核电运营", "核能发电")),
|
||||
("CHAIN_UTIL_MID_POWER_RENEW", ("风力发电运营", "光伏发电运营", "新能源发电", "可再生能源发电")),
|
||||
("CHAIN_UTIL_MID_CHP", ("热电联产", "集中供热", "蒸汽供应", "供热业务", "电力、热力的生产与销售", "生产、销售电力热力产品")),
|
||||
("CHAIN_UTIL_MID_GRID", ("配电网运营", "电网运营", "增量配电业务")),
|
||||
("CHAIN_UTIL_MID_GAS_PIPE", ("天然气长输管道", "天然气管网运营", "管道天然气输送")),
|
||||
("CHAIN_UTIL_MID_GAS_CITY", ("城市燃气", "城镇燃气", "燃气管网运营")),
|
||||
("CHAIN_UTIL_DOWN_RETAIL", ("电力销售", "发电业务并销售电力", "生产和销售电力", "售电业务", "综合能源服务", "能源托管服务")),
|
||||
("CHAIN_UTIL_DOWN_GAS", ("天然气销售", "燃气销售", "燃气接驳", "燃气增值服务")),
|
||||
)
|
||||
|
||||
|
||||
ENVIRONMENT_DEFINITIONS = (
|
||||
("CHAIN_ENV", "环保产业链", "", 1),
|
||||
("CHAIN_ENV_UP", "上游设备与监测", "CHAIN_ENV", 2),
|
||||
("CHAIN_ENV_UP_WATER_EQ", "水处理设备与膜", "CHAIN_ENV_UP", 3),
|
||||
("CHAIN_ENV_UP_AIR_EQ", "大气治理设备", "CHAIN_ENV_UP", 3),
|
||||
("CHAIN_ENV_UP_SOLID_EQ", "固废与环卫装备", "CHAIN_ENV_UP", 3),
|
||||
("CHAIN_ENV_UP_MONITOR", "环境监测", "CHAIN_ENV_UP", 3),
|
||||
("CHAIN_ENV_MID", "中游治理工程与服务", "CHAIN_ENV", 2),
|
||||
("CHAIN_ENV_MID_WATER", "水环境治理", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_MID_AIR", "大气污染治理", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_MID_WASTE", "固废处理", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_MID_WASTE_INCIN", "垃圾焚烧", "CHAIN_ENV_MID_WASTE", 4),
|
||||
("CHAIN_ENV_MID_WASTE_HAZ", "危废与医废", "CHAIN_ENV_MID_WASTE", 4),
|
||||
("CHAIN_ENV_MID_SANITATION", "环卫服务", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_MID_SOIL", "土壤与生态修复", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_MID_RECYCLE", "资源循环利用", "CHAIN_ENV_MID", 3),
|
||||
("CHAIN_ENV_DOWN", "下游设施运营", "CHAIN_ENV", 2),
|
||||
("CHAIN_ENV_DOWN_SEWAGE", "污水处理运营", "CHAIN_ENV_DOWN", 3),
|
||||
("CHAIN_ENV_DOWN_WASTE", "固废设施运营", "CHAIN_ENV_DOWN", 3),
|
||||
("CHAIN_ENV_DOWN_ENERGY", "节能与碳服务", "CHAIN_ENV_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
ENVIRONMENT_RULES = (
|
||||
("CHAIN_ENV_UP_WATER_EQ", ("水处理设备", "污水处理设备", "膜分离设备", "水处理膜", "反渗透膜")),
|
||||
("CHAIN_ENV_UP_AIR_EQ", ("烟气治理设备", "脱硫脱硝设备", "除尘设备", "挥发性有机物治理设备")),
|
||||
("CHAIN_ENV_UP_SOLID_EQ", ("环卫装备", "垃圾处理设备", "固废处理设备", "环卫车辆")),
|
||||
("CHAIN_ENV_UP_MONITOR", ("环境监测仪器", "环境监测设备", "污染源在线监测", "环境检测服务")),
|
||||
("CHAIN_ENV_MID_WATER", ("水环境综合治理", "污水处理工程", "河湖治理", "工业废水治理")),
|
||||
("CHAIN_ENV_MID_AIR", ("大气污染治理", "烟气脱硫脱硝", "工业废气治理", "vocs治理")),
|
||||
("CHAIN_ENV_MID_WASTE_INCIN", ("垃圾焚烧发电", "生活垃圾焚烧", "垃圾焚烧处理")),
|
||||
("CHAIN_ENV_MID_WASTE_HAZ", ("危险废物处置", "危废处理", "医疗废物处置")),
|
||||
("CHAIN_ENV_MID_SANITATION", ("环卫服务", "城乡环卫", "道路清扫保洁", "垃圾收运")),
|
||||
("CHAIN_ENV_MID_SOIL", ("土壤修复", "矿山生态修复", "场地修复", "生态环境修复")),
|
||||
("CHAIN_ENV_MID_RECYCLE", ("再生资源回收", "资源循环利用", "废旧物资回收", "再生金属利用")),
|
||||
("CHAIN_ENV_DOWN_SEWAGE", ("污水处理运营", "污水处理厂运营", "供排水运营")),
|
||||
("CHAIN_ENV_DOWN_WASTE", ("固废处理设施运营", "垃圾处理厂运营", "垃圾焚烧项目运营")),
|
||||
("CHAIN_ENV_DOWN_ENERGY", ("合同能源管理", "节能服务", "碳资产管理", "碳咨询服务")),
|
||||
)
|
||||
|
||||
|
||||
REAL_ESTATE_DEFINITIONS = (
|
||||
("CHAIN_RE", "房地产产业链", "", 1),
|
||||
("CHAIN_RE_UP", "上游开发准备", "CHAIN_RE", 2),
|
||||
("CHAIN_RE_UP_PLANNING", "规划设计与代建", "CHAIN_RE_UP", 3),
|
||||
("CHAIN_RE_UP_RENEWAL", "城市更新与旧改", "CHAIN_RE_UP", 3),
|
||||
("CHAIN_RE_MID", "中游开发建设", "CHAIN_RE", 2),
|
||||
("CHAIN_RE_MID_INTEGRATED", "综合房地产开发", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_MID_RESIDENTIAL", "住宅开发", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_MID_COMMERCIAL", "商业地产开发", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_MID_OFFICE", "写字楼与综合体开发", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_MID_PARK", "产业园区与物流地产", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_MID_TOUR", "文旅与康养地产", "CHAIN_RE_MID", 3),
|
||||
("CHAIN_RE_DOWN", "下游持有运营与服务", "CHAIN_RE", 2),
|
||||
("CHAIN_RE_DOWN_PROPERTY", "物业管理", "CHAIN_RE_DOWN", 3),
|
||||
("CHAIN_RE_DOWN_COMMERCIAL", "商业运营", "CHAIN_RE_DOWN", 3),
|
||||
("CHAIN_RE_DOWN_RENTAL", "租赁住房与公寓运营", "CHAIN_RE_DOWN", 3),
|
||||
("CHAIN_RE_DOWN_BROKER", "房地产经纪与交易服务", "CHAIN_RE_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
REAL_ESTATE_RULES = (
|
||||
("CHAIN_RE_UP_PLANNING", ("房地产代建", "商业代建", "项目代建管理", "房地产规划设计")),
|
||||
("CHAIN_RE_UP_RENEWAL", ("城市更新项目开发", "旧城改造项目", "城市更新运营")),
|
||||
("CHAIN_RE_MID_RESIDENTIAL", ("住宅地产开发", "商品住宅开发", "住宅开发销售", "房地产开发与商品房销售")),
|
||||
("CHAIN_RE_MID_COMMERCIAL", ("商业地产开发", "购物中心开发", "商业物业开发")),
|
||||
("CHAIN_RE_MID_OFFICE", ("写字楼开发", "城市综合体开发", "商务园区开发")),
|
||||
("CHAIN_RE_MID_PARK", ("产业园区开发运营", "物流地产开发", "工业园区开发运营")),
|
||||
("CHAIN_RE_MID_TOUR", ("文旅地产开发", "康养地产开发", "旅游地产开发")),
|
||||
("CHAIN_RE_DOWN_PROPERTY", ("物业管理服务", "住宅物业管理", "商业物业管理")),
|
||||
("CHAIN_RE_DOWN_COMMERCIAL", ("购物中心运营", "商业物业运营", "商业运营管理")),
|
||||
("CHAIN_RE_DOWN_RENTAL", ("长租公寓运营", "租赁住房运营", "保障性租赁住房运营")),
|
||||
("CHAIN_RE_DOWN_BROKER", ("房地产经纪服务", "房产中介服务", "房地产交易服务")),
|
||||
("CHAIN_RE_MID_INTEGRATED", ("房地产开发", "商品房开发", "地产开发", "房地产开发经营", "房产开发")),
|
||||
)
|
||||
|
||||
|
||||
BUILDING_MATERIAL_DEFINITIONS = (
|
||||
("CHAIN_MAT", "建筑材料产业链", "", 1),
|
||||
("CHAIN_MAT_UP", "上游矿物与原料", "CHAIN_MAT", 2),
|
||||
("CHAIN_MAT_UP_AGGREGATE", "砂石骨料", "CHAIN_MAT_UP", 3),
|
||||
("CHAIN_MAT_UP_ADMIXTURE", "外加剂与建材化学品", "CHAIN_MAT_UP", 3),
|
||||
("CHAIN_MAT_MID", "中游基础建材", "CHAIN_MAT", 2),
|
||||
("CHAIN_MAT_MID_CEMENT", "水泥与熟料", "CHAIN_MAT_MID", 3),
|
||||
("CHAIN_MAT_MID_CONCRETE", "混凝土与水泥制品", "CHAIN_MAT_MID", 3),
|
||||
("CHAIN_MAT_MID_GLASS", "建筑玻璃", "CHAIN_MAT_MID", 3),
|
||||
("CHAIN_MAT_MID_FIBER", "玻璃纤维与复合材料", "CHAIN_MAT_MID", 3),
|
||||
("CHAIN_MAT_MID_CERAMIC", "建筑陶瓷与耐火材料", "CHAIN_MAT_MID", 3),
|
||||
("CHAIN_MAT_DOWN", "下游装修建材", "CHAIN_MAT", 2),
|
||||
("CHAIN_MAT_DOWN_WATERPROOF", "防水材料", "CHAIN_MAT_DOWN", 3),
|
||||
("CHAIN_MAT_DOWN_PIPE", "建筑管材", "CHAIN_MAT_DOWN", 3),
|
||||
("CHAIN_MAT_DOWN_BOARD", "石膏板与板材", "CHAIN_MAT_DOWN", 3),
|
||||
("CHAIN_MAT_DOWN_COATING", "建筑涂料", "CHAIN_MAT_DOWN", 3),
|
||||
("CHAIN_MAT_DOWN_HARDWARE", "门窗幕墙与五金", "CHAIN_MAT_DOWN", 3),
|
||||
("CHAIN_MAT_DOWN_NEW", "电子与新能源建材", "CHAIN_MAT_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
BUILDING_MATERIAL_RULES = (
|
||||
("CHAIN_MAT_UP_AGGREGATE", ("砂石骨料开采", "建筑石料开采", "机制砂生产", "骨料生产销售")),
|
||||
("CHAIN_MAT_UP_ADMIXTURE", ("混凝土外加剂", "减水剂生产", "建筑胶黏剂", "建材化学品")),
|
||||
("CHAIN_MAT_MID_CEMENT", ("水泥熟料生产", "水泥生产销售", "水泥、建材", "水泥、水泥制品", "硅酸盐水泥", "特种水泥")),
|
||||
("CHAIN_MAT_MID_CONCRETE", ("商品混凝土生产", "预拌混凝土", "混凝土构件", "水泥制品生产", "预应力混凝土", "预应力钢筒混凝土管", "钢筋混凝土排水管", "高端混凝土制品")),
|
||||
("CHAIN_MAT_MID_GLASS", ("浮法玻璃生产", "建筑玻璃生产", "工程玻璃", "节能玻璃", "低辐射镀膜玻璃")),
|
||||
("CHAIN_MAT_MID_FIBER", ("玻璃纤维生产", "玻璃纤维及其制品", "特种玻璃纤维", "玻纤纱", "玻纤织物", "玻璃纤维复合材料", "玻璃纤维增强复合材料", "电子级玻纤布")),
|
||||
("CHAIN_MAT_MID_CERAMIC", ("建筑陶瓷生产", "瓷砖生产", "耐火材料生产", "耐火材料", "耐磨材料", "陶瓷砖", "陶瓷纤维")),
|
||||
("CHAIN_MAT_DOWN_WATERPROOF", ("建筑防水材料", "防水卷材", "防水涂料生产", "防水工程材料")),
|
||||
("CHAIN_MAT_DOWN_PIPE", ("建筑塑料管道", "新型塑料管道", "给排水管材", "建筑管材生产", "塑料管材管件")),
|
||||
("CHAIN_MAT_DOWN_BOARD", ("石膏板生产", "纸面石膏板", "建筑板材", "装饰板材", "人造石英石", "纤维水泥板")),
|
||||
("CHAIN_MAT_DOWN_COATING", ("建筑涂料生产", "墙面涂料", "工程涂料", "保温装饰涂料")),
|
||||
("CHAIN_MAT_DOWN_HARDWARE", ("建筑门窗生产", "系统门窗", "智慧幕墙系统", "建筑幕墙产品", "铝合金门窗", "门窗五金", "建筑五金")),
|
||||
("CHAIN_MAT_DOWN_NEW", ("电子玻纤布", "光伏玻璃生产", "光伏压延玻璃", "锂电玻纤材料")),
|
||||
)
|
||||
|
||||
|
||||
FINANCE_DEFINITIONS = (
|
||||
("CHAIN_FIN", "金融服务产业链", "", 1),
|
||||
("CHAIN_FIN_BANK", "银行服务", "CHAIN_FIN", 2),
|
||||
("CHAIN_FIN_BANK_STATE", "综合商业银行", "CHAIN_FIN_BANK", 3),
|
||||
("CHAIN_FIN_BANK_CITY", "城商行", "CHAIN_FIN_BANK", 3),
|
||||
("CHAIN_FIN_BANK_RURAL", "农商行", "CHAIN_FIN_BANK", 3),
|
||||
("CHAIN_FIN_SEC", "证券与资本市场", "CHAIN_FIN", 2),
|
||||
("CHAIN_FIN_SEC_INTEGRATED", "综合证券公司", "CHAIN_FIN_SEC", 3),
|
||||
("CHAIN_FIN_SEC_BROKER", "证券经纪与财富管理", "CHAIN_FIN_SEC", 3),
|
||||
("CHAIN_FIN_SEC_IB", "投资银行", "CHAIN_FIN_SEC", 3),
|
||||
("CHAIN_FIN_SEC_ASSET", "资管与基金", "CHAIN_FIN_SEC", 3),
|
||||
("CHAIN_FIN_SEC_FUTURES", "期货与衍生品", "CHAIN_FIN_SEC", 3),
|
||||
("CHAIN_FIN_INS", "保险服务", "CHAIN_FIN", 2),
|
||||
("CHAIN_FIN_INS_GROUP", "综合保险集团", "CHAIN_FIN_INS", 3),
|
||||
("CHAIN_FIN_INS_LIFE", "人身险", "CHAIN_FIN_INS", 3),
|
||||
("CHAIN_FIN_INS_PNC", "财产险", "CHAIN_FIN_INS", 3),
|
||||
("CHAIN_FIN_INS_RE", "再保险", "CHAIN_FIN_INS", 3),
|
||||
("CHAIN_FIN_MULTI", "多元金融", "CHAIN_FIN", 2),
|
||||
("CHAIN_FIN_MULTI_TRUST", "信托", "CHAIN_FIN_MULTI", 3),
|
||||
("CHAIN_FIN_MULTI_LEASE", "融资租赁", "CHAIN_FIN_MULTI", 3),
|
||||
("CHAIN_FIN_MULTI_FACTOR", "保理与担保", "CHAIN_FIN_MULTI", 3),
|
||||
("CHAIN_FIN_MULTI_AMC", "资产管理与不良处置", "CHAIN_FIN_MULTI", 3),
|
||||
)
|
||||
|
||||
|
||||
FINANCE_RULES = (
|
||||
("CHAIN_FIN_BANK_STATE", ("公司和个人金融业务", "普惠金融业务、公司金融业务", "对公及对私存款", "存贷款业务", "公司业务、个人业务", "全国性商业银行", "股份制商业银行", "国有大型商业银行", "商业银行及相关金融服务", "商业银行业务", "商业银行", "公司银行业务", "个人银行业务", "零售银行业务", "银行业务")),
|
||||
("CHAIN_FIN_BANK_CITY", ("城市商业银行", "城商行业务")),
|
||||
("CHAIN_FIN_BANK_RURAL", ("农村商业银行", "农商行业务")),
|
||||
("CHAIN_FIN_SEC_BROKER", ("证券经纪业务", "财富管理业务", "证券投资咨询")),
|
||||
("CHAIN_FIN_SEC_IB", ("投资银行业务", "证券承销保荐", "并购重组财务顾问")),
|
||||
("CHAIN_FIN_SEC_ASSET", ("证券资产管理", "公募基金管理", "私募资产管理", "私募股权投资基金管理", "投资管理业务")),
|
||||
("CHAIN_FIN_SEC_FUTURES", ("期货经纪业务", "期货交易服务", "风险管理业务", "风险管理子公司业务")),
|
||||
("CHAIN_FIN_SEC_INTEGRATED", ("证券及期货经纪、证券投资", "证券公司业务", "综合证券业务", "企业金融、个人金融、机构服务")),
|
||||
("CHAIN_FIN_INS_GROUP", ("综合保险集团", "寿险及健康险业务、财产保险业务", "人身保险产品和服务、财产保险产品和服务", "各类风险保障")),
|
||||
("CHAIN_FIN_INS_LIFE", ("人寿保险业务", "个人人寿保险", "团体人寿保险", "健康保险业务", "健康险等产品与服务", "年金保险业务")),
|
||||
("CHAIN_FIN_INS_PNC", ("财产保险业务", "机动车辆保险", "责任保险业务")),
|
||||
("CHAIN_FIN_INS_RE", ("再保险业务", "人身再保险", "财产再保险")),
|
||||
("CHAIN_FIN_MULTI_TRUST", ("信托业务", "信托资产管理")),
|
||||
("CHAIN_FIN_MULTI_LEASE", ("融资租赁业务", "金融租赁业务")),
|
||||
("CHAIN_FIN_MULTI_FACTOR", ("商业保理业务", "融资担保业务", "信用担保业务")),
|
||||
("CHAIN_FIN_MULTI_AMC", ("不良资产处置", "地方资产管理公司", "特殊资产管理")),
|
||||
)
|
||||
|
||||
|
||||
BUSINESS_INDUSTRY_DEFINITIONS = (
|
||||
("CHAIN_BIZ", "F10主营产业链", "", 1),
|
||||
("CHAIN_BIZ_HEALTH", "医药与生命科学", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_MATERIAL", "先进材料", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_EQUIPMENT", "工业装备与检测", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_AUTO", "汽车零部件", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_ENERGY", "能源与电力", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_MINING", "矿产与金属", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_ELECTRONIC", "电子信息制造", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_SOFTWARE", "软件与数字服务", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_CONSUMER", "消费品与零售", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_AGRI_FOOD", "农业与食品", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_TRANSPORT", "交通运输与物流", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_CONSTRUCTION", "建筑工程与建材", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_FINANCE", "金融与投资", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_MEDIA", "文化传媒与游戏", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_CHEMICAL", "化工与高分子", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_SERVICE", "专业与公共服务", "CHAIN_BIZ", 2),
|
||||
("CHAIN_BIZ_MULTI", "多元综合业务", "CHAIN_BIZ", 2),
|
||||
)
|
||||
|
||||
|
||||
BUSINESS_INDUSTRY_RULES = (
|
||||
("CHAIN_BIZ_HEALTH", ("药品", "药物", "医药", "生物制剂", "诊断试剂", "医疗器械", "生命科学", "临床研究")),
|
||||
("CHAIN_BIZ_AUTO", ("汽车零部件", "汽车电子", "智能座舱", "发动机缸体", "汽车内饰", "汽车外饰", "汽车安全部件")),
|
||||
("CHAIN_BIZ_ELECTRONIC", ("电子元器件", "电子产品", "显示面板", "显示器件", "印制电路板", "覆铜板", "半导体", "芯片", "led", "通信设备")),
|
||||
("CHAIN_BIZ_SOFTWARE", ("软件产品", "软件开发", "信息技术服务", "数据服务", "数字化服务", "云服务", "信息服务", "系统集成")),
|
||||
("CHAIN_BIZ_MEDIA", ("游戏", "影视", "广告传媒", "数字阅读", "出版", "短剧", "文化展示")),
|
||||
("CHAIN_BIZ_FINANCE", ("银行业务", "保险业务", "人寿保险", "证券业务", "创业投资", "资产管理", "金融服务")),
|
||||
("CHAIN_BIZ_AGRI_FOOD", ("种植", "养殖", "种鸡", "种猪", "食品", "肉制品", "乳制品", "白砂糖", "农产品")),
|
||||
("CHAIN_BIZ_TRANSPORT", ("航空运输", "道路运输", "航运", "港口", "物流配送", "仓储物流", "快递")),
|
||||
("CHAIN_BIZ_CONSTRUCTION", ("建筑工程", "工程施工", "工程咨询", "幕墙和门窗", "输水管道", "建材", "水泥", "玻璃纤维")),
|
||||
("CHAIN_BIZ_ENERGY", ("电力", "热力", "发电", "煤炭", "天然气", "lng", "能源服务")),
|
||||
("CHAIN_BIZ_MINING", ("矿产资源", "矿石开采", "采矿", "选矿", "冶炼", "贵金属", "有色金属", "稀土", "黄金", "白银")),
|
||||
("CHAIN_BIZ_CHEMICAL", ("基础化工", "化学纤维", "化学品", "化工产品", "树脂", "烧碱", "涂料", "胶粘剂", "农药", "染料", "助剂", "高分子")),
|
||||
("CHAIN_BIZ_MATERIAL", ("新材料", "复合材料", "金刚石", "超硬材料", "磁性材料", "功能性薄膜", "光学膜", "陶瓷", "合金加工材", "纤维材料")),
|
||||
("CHAIN_BIZ_EQUIPMENT", ("机械设备", "通用设备", "设备的研发", "装备的研发", "专用设备", "压缩机", "真空泵", "自动化设备", "检测服务", "解决方案")),
|
||||
("CHAIN_BIZ_CONSUMER", ("零售业", "超市", "百货", "家用电器", "家居产品", "玩具", "服装", "酒店运营")),
|
||||
("CHAIN_BIZ_SERVICE", ("专业服务", "咨询服务", "技术服务", "物业管理", "园区运营")),
|
||||
)
|
||||
@@ -0,0 +1,583 @@
|
||||
"""Industry definitions and product rules: technology."""
|
||||
|
||||
SEMICONDUCTOR_DEFINITIONS = (
|
||||
("CHAIN_SEMI", "半导体产业链", "", 1),
|
||||
("CHAIN_SEMI_UP", "上游支撑", "CHAIN_SEMI", 2),
|
||||
("CHAIN_SEMI_UP_MATERIAL", "半导体材料", "CHAIN_SEMI_UP", 3),
|
||||
("CHAIN_SEMI_UP_MAT_WAFER", "硅片与晶圆材料", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_PHOTO", "光刻胶与湿电子化学品", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_GAS", "电子特气", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_CMP", "靶材与CMP材料", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_MASK", "光掩模", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_PACKAGE", "封装材料", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_MAT_WBG", "第三代半导体材料", "CHAIN_SEMI_UP_MATERIAL", 4),
|
||||
("CHAIN_SEMI_UP_EQUIPMENT", "半导体设备", "CHAIN_SEMI_UP", 3),
|
||||
("CHAIN_SEMI_UP_EQ_LITHO", "光刻与涂胶显影", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
# Retain the old node for existing memberships until their evidence refresh succeeds.
|
||||
# No classification rule emits this combined code anymore.
|
||||
("CHAIN_SEMI_UP_EQ_ETCH", "刻蚀与薄膜沉积", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_ETCHING", "刻蚀设备", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_DEPOSITION", "薄膜沉积设备", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_IMPLANT", "离子注入与热处理", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_CLEAN", "清洗与CMP设备", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_METROLOGY", "量测与缺陷检测", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_TEST", "测试设备", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EQ_PACKAGE", "封装设备", "CHAIN_SEMI_UP_EQUIPMENT", 4),
|
||||
("CHAIN_SEMI_UP_EDA", "EDA/IP", "CHAIN_SEMI_UP", 3),
|
||||
("CHAIN_SEMI_UP_EDA_SOFTWARE", "EDA软件", "CHAIN_SEMI_UP_EDA", 4),
|
||||
("CHAIN_SEMI_UP_EDA_IP", "半导体IP", "CHAIN_SEMI_UP_EDA", 4),
|
||||
("CHAIN_SEMI_MID", "中游制造", "CHAIN_SEMI", 2),
|
||||
("CHAIN_SEMI_MID_DESIGN", "芯片设计", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_MID_DESIGN_MEMORY", "存储芯片", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_DESIGN_ANALOG", "模拟芯片", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_DESIGN_DIGITAL", "数字芯片", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_DESIGN_POWER", "功率芯片", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_DESIGN_SENSOR", "传感/光电芯片", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_DESIGN_OTHER", "其他芯片设计", "CHAIN_SEMI_MID_DESIGN", 4),
|
||||
("CHAIN_SEMI_MID_FOUNDRY", "晶圆制造", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_MID_PACKAGE", "封装测试", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_MID_PACKAGE_TRAD", "传统封装", "CHAIN_SEMI_MID_PACKAGE", 4),
|
||||
("CHAIN_SEMI_MID_PACKAGE_ADV", "先进封装", "CHAIN_SEMI_MID_PACKAGE", 4),
|
||||
("CHAIN_SEMI_MID_PACKAGE_TEST", "晶圆与成品测试", "CHAIN_SEMI_MID_PACKAGE", 4),
|
||||
("CHAIN_SEMI_MID_IDM", "IDM/分立器件", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_MID_DISPLAY", "半导体显示与面板", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_MID_OTHER", "半导体综合", "CHAIN_SEMI_MID", 3),
|
||||
("CHAIN_SEMI_DOWN", "下游应用", "CHAIN_SEMI", 2),
|
||||
("CHAIN_SEMI_DOWN_AUTO", "汽车电子", "CHAIN_SEMI_DOWN", 3),
|
||||
("CHAIN_SEMI_DOWN_CONSUMER", "消费电子", "CHAIN_SEMI_DOWN", 3),
|
||||
("CHAIN_SEMI_DOWN_COMM", "通信与物联网", "CHAIN_SEMI_DOWN", 3),
|
||||
("CHAIN_SEMI_DOWN_INDUSTRIAL", "工业与能源", "CHAIN_SEMI_DOWN", 3),
|
||||
("CHAIN_SEMI_DOWN_AI", "AI与数据中心", "CHAIN_SEMI_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
SEMICONDUCTOR_RULES = (
|
||||
("CHAIN_SEMI_UP_EQ_LITHO", ("光刻机", "涂胶显影", "光刻设备")),
|
||||
("CHAIN_SEMI_UP_EQ_ETCHING", ("刻蚀机", "刻蚀设备")),
|
||||
("CHAIN_SEMI_UP_EQ_DEPOSITION", ("薄膜沉积设备", "薄膜沉积", "薄膜设备", "pecvd", "lpcvd", "ald设备", "cvd设备", "pvd设备", "原子层沉积设备", "物理气相沉积设备", "化学气相沉积设备", "原子层沉积", "物理气相沉积", "化学气相沉积")),
|
||||
("CHAIN_SEMI_UP_EQ_IMPLANT", ("离子注入", "扩散炉", "热处理设备", "退火设备")),
|
||||
("CHAIN_SEMI_UP_EQ_CLEAN", ("清洗设备", "cmp设备", "化学机械抛光设备")),
|
||||
("CHAIN_SEMI_UP_EQ_METROLOGY", ("量测设备", "缺陷检测", "晶圆检测设备", "半导体检测设备")),
|
||||
("CHAIN_SEMI_UP_EQ_TEST", ("半导体测试机", "探针台", "分选机", "测试设备")),
|
||||
("CHAIN_SEMI_UP_EQ_PACKAGE", ("封装设备", "固晶机", "键合机", "划片机")),
|
||||
("CHAIN_SEMI_UP_EQUIPMENT", ("半导体设备", "半导体装备", "集成电路设备", "电子工艺装备")),
|
||||
("CHAIN_SEMI_UP_MAT_WAFER", ("半导体硅片", "晶圆材料", "大硅片", "外延片")),
|
||||
("CHAIN_SEMI_UP_MAT_PHOTO", ("光刻胶", "湿电子化学品", "显影液", "剥离液", "电子级化学品")),
|
||||
("CHAIN_SEMI_UP_MAT_GAS", ("电子特气", "半导体特气", "电子气体")),
|
||||
("CHAIN_SEMI_UP_MAT_CMP", ("半导体靶材", "cmp材料", "抛光液", "抛光垫")),
|
||||
("CHAIN_SEMI_UP_MAT_MASK", ("光掩模", "掩膜版", "光罩")),
|
||||
("CHAIN_SEMI_UP_MAT_PACKAGE", ("封装材料", "引线框架", "键合丝", "封装基板", "环氧塑封料")),
|
||||
("CHAIN_SEMI_UP_MAT_WBG", ("碳化硅衬底", "氮化镓衬底", "第三代半导体材料")),
|
||||
("CHAIN_SEMI_UP_MATERIAL", ("半导体材料",)),
|
||||
("CHAIN_SEMI_UP_EDA_SOFTWARE", ("eda软件", "集成电路设计软件", "电子设计自动化")),
|
||||
("CHAIN_SEMI_UP_EDA_IP", ("ip授权", "ip核", "知识产权核", "半导体ip")),
|
||||
("CHAIN_SEMI_MID_PACKAGE_ADV", ("先进封装", "chiplet", "fan-out", "扇出封装", "2.5d封装", "3d封装", "cowos", "系统级封装", "sip封装")),
|
||||
("CHAIN_SEMI_MID_PACKAGE_TEST", ("晶圆测试", "成品测试", "测试服务")),
|
||||
("CHAIN_SEMI_MID_PACKAGE_TRAD", ("传统封装", "引线键合封装")),
|
||||
("CHAIN_SEMI_MID_PACKAGE", ("封装测试", "封装、测试", "集成电路封测", "封测服务")),
|
||||
("CHAIN_SEMI_MID_FOUNDRY", ("晶圆代工", "晶圆制造", "集成电路晶圆", "芯片制造")),
|
||||
("CHAIN_SEMI_MID_IDM", ("功率器件制造", "分立器件制造", "半导体分立器件", "idm")),
|
||||
("CHAIN_SEMI_MID_DISPLAY", ("半导体显示", "显示器件业务", "液晶显示面板", "tft-lcd", "amoled显示", "mled业务")),
|
||||
("CHAIN_SEMI_MID_DESIGN_MEMORY", ("存储器", "存储芯片", "nor flash", "nand", "dram", "eeprom")),
|
||||
("CHAIN_SEMI_MID_DESIGN_ANALOG", ("模拟芯片", "模拟集成电路", "电源管理芯片", "信号链")),
|
||||
("CHAIN_SEMI_MID_DESIGN_POWER", ("功率芯片", "功率半导体", "igbt", "mosfet", "碳化硅芯片", "氮化镓芯片")),
|
||||
("CHAIN_SEMI_MID_DESIGN_SENSOR", ("传感器芯片", "mems", "光芯片", "射频芯片", "图像传感器")),
|
||||
("CHAIN_SEMI_MID_DESIGN_DIGITAL", ("数字芯片", "微控制器", "mcu", "cpu", "gpu", "soc", "fpga", "asic", "处理器芯片")),
|
||||
("CHAIN_SEMI_MID_DESIGN_OTHER", ("芯片设计", "集成电路设计", "芯片研发", "集成电路研发")),
|
||||
("CHAIN_SEMI_DOWN_AUTO", ("汽车电子系统", "车载终端", "智能座舱系统")),
|
||||
("CHAIN_SEMI_DOWN_CONSUMER", ("消费电子整机", "智能终端整机", "可穿戴设备")),
|
||||
("CHAIN_SEMI_DOWN_COMM", ("通信终端", "物联网终端", "通信系统设备")),
|
||||
("CHAIN_SEMI_DOWN_INDUSTRIAL", ("工业控制系统", "新能源控制系统", "电力电子系统")),
|
||||
("CHAIN_SEMI_DOWN_AI", ("人工智能服务器", "数据中心设备", "ai服务器")),
|
||||
)
|
||||
|
||||
|
||||
ROBOT_DEFINITIONS = (
|
||||
("CHAIN_ROBOT", "机器人产业链", "", 1),
|
||||
("CHAIN_ROBOT_UP", "上游软硬件与核心部件", "CHAIN_ROBOT", 2),
|
||||
("CHAIN_ROBOT_UP_MATERIAL", "材料与结构件", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_REDUCER", "精密减速器", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_SERVO", "电机与伺服系统", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_CONTROL", "控制器与算力芯片", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_SENSOR", "感知系统", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_SENSOR_VISION", "机器视觉与3D视觉", "CHAIN_ROBOT_UP_SENSOR", 4),
|
||||
("CHAIN_ROBOT_UP_SENSOR_FORCE", "力矩与触觉传感", "CHAIN_ROBOT_UP_SENSOR", 4),
|
||||
("CHAIN_ROBOT_UP_SENSOR_POSITION", "位置与惯性传感", "CHAIN_ROBOT_UP_SENSOR", 4),
|
||||
("CHAIN_ROBOT_UP_TRANSMISSION", "丝杠/轴承/导轨", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_JOINT", "关节与执行器总成", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_HAND", "灵巧手", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_BATTERY", "电池与电源系统", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_UP_AI", "具身模型/数据/仿真", "CHAIN_ROBOT_UP", 3),
|
||||
("CHAIN_ROBOT_MID", "中游本体与集成", "CHAIN_ROBOT", 2),
|
||||
("CHAIN_ROBOT_MID_HUMANOID", "人形机器人", "CHAIN_ROBOT_MID", 3),
|
||||
("CHAIN_ROBOT_MID_INDUSTRIAL", "工业机器人", "CHAIN_ROBOT_MID", 3),
|
||||
("CHAIN_ROBOT_MID_SERVICE", "服务机器人", "CHAIN_ROBOT_MID", 3),
|
||||
("CHAIN_ROBOT_MID_SPECIAL", "特种机器人", "CHAIN_ROBOT_MID", 3),
|
||||
("CHAIN_ROBOT_MID_INTEGRATION", "系统集成", "CHAIN_ROBOT_MID", 3),
|
||||
("CHAIN_ROBOT_DOWN", "下游应用", "CHAIN_ROBOT", 2),
|
||||
("CHAIN_ROBOT_DOWN_AUTO", "汽车制造", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_ELECTRONICS", "电子制造", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_LOGISTICS", "物流仓储", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_MEDICAL", "医疗与服务", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_RESEARCH", "科研教育", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_HOME", "商业与家庭服务", "CHAIN_ROBOT_DOWN", 3),
|
||||
("CHAIN_ROBOT_DOWN_SPECIAL", "巡检与特种作业", "CHAIN_ROBOT_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
ROBOT_RULES = (
|
||||
("CHAIN_ROBOT_UP_MATERIAL", ("机器人结构件", "人形机器人等智能终端的结构件", "人形机器人结构件", "轻量化材料", "铝合金压铸件", "碳纤维结构件")),
|
||||
("CHAIN_ROBOT_UP_REDUCER", ("谐波减速器", "rv减速器", "精密减速器", "精密减速机", "机器人减速器")),
|
||||
("CHAIN_ROBOT_UP_SERVO", ("伺服电机", "伺服系统", "空心杯电机", "无框力矩电机", "步进电机", "机器人电机")),
|
||||
("CHAIN_ROBOT_UP_CONTROL", ("机器人控制器", "运动控制器", "运动控制系统", "机器人控制系统", "机器人芯片", "边缘计算模块")),
|
||||
("CHAIN_ROBOT_UP_SENSOR_VISION", ("机器视觉", "视觉系统", "3d视觉", "深度相机", "激光雷达")),
|
||||
("CHAIN_ROBOT_UP_SENSOR_FORCE", ("力矩传感器", "六维力传感器", "触觉传感器", "电子皮肤")),
|
||||
("CHAIN_ROBOT_UP_SENSOR_POSITION", ("编码器", "惯性传感器", "imu", "位置传感器")),
|
||||
("CHAIN_ROBOT_UP_SENSOR", ("机器人传感器", "感知系统")),
|
||||
("CHAIN_ROBOT_UP_TRANSMISSION", ("滚珠丝杠", "行星滚柱丝杠", "精密轴承", "直线导轨", "机器人关节")),
|
||||
("CHAIN_ROBOT_UP_JOINT", ("一体化关节", "关节模组", "执行器总成", "旋转执行器", "线性执行器")),
|
||||
("CHAIN_ROBOT_UP_HAND", ("灵巧手", "仿生手", "机器人手指")),
|
||||
("CHAIN_ROBOT_UP_BATTERY", ("机器人电池", "机器人电源", "高倍率电池")),
|
||||
("CHAIN_ROBOT_UP_AI", ("具身大模型", "具身智能模型", "机器人训练数据", "机器人仿真平台", "动作数据集")),
|
||||
("CHAIN_ROBOT_MID_HUMANOID", ("人形机器人", "具身智能机器人")),
|
||||
("CHAIN_ROBOT_MID_INDUSTRIAL", ("工业机器人", "协作机器人", "焊接机器人", "机器人本体", "机械臂")),
|
||||
("CHAIN_ROBOT_MID_SERVICE", ("服务机器人", "清洁机器人", "扫地机器人", "医疗机器人", "康复机器人")),
|
||||
("CHAIN_ROBOT_MID_SPECIAL", ("特种机器人", "消防机器人", "巡检机器人", "水下机器人", "排爆机器人")),
|
||||
("CHAIN_ROBOT_MID_INTEGRATION", ("机器人系统集成", "自动化生产线", "智能制造系统", "工业自动化装备")),
|
||||
("CHAIN_ROBOT_DOWN_AUTO", ("汽车焊装线", "汽车制造自动化", "汽车装配线")),
|
||||
("CHAIN_ROBOT_DOWN_ELECTRONICS", ("3c自动化", "电子制造自动化", "半导体搬运系统")),
|
||||
("CHAIN_ROBOT_DOWN_LOGISTICS", ("物流机器人", "仓储机器人", "智能仓储系统", "agv")),
|
||||
("CHAIN_ROBOT_DOWN_MEDICAL", ("手术机器人", "医疗服务机器人", "养老机器人")),
|
||||
("CHAIN_ROBOT_DOWN_RESEARCH", ("科研教育机器人", "机器人科研平台", "教育机器人")),
|
||||
("CHAIN_ROBOT_DOWN_HOME", ("家庭服务机器人", "导览机器人", "导购机器人", "家用机器人")),
|
||||
("CHAIN_ROBOT_DOWN_SPECIAL", ("电力巡检机器人", "矿山机器人", "消防机器人", "排爆机器人")),
|
||||
)
|
||||
|
||||
|
||||
SPACE_DEFINITIONS = (
|
||||
("CHAIN_SPACE", "商业航天产业链", "", 1),
|
||||
("CHAIN_SPACE_UP", "上游材料与部件", "CHAIN_SPACE", 2),
|
||||
("CHAIN_SPACE_UP_MATERIAL", "航天材料", "CHAIN_SPACE_UP", 3),
|
||||
("CHAIN_SPACE_UP_PART", "核心零部件", "CHAIN_SPACE_UP", 3),
|
||||
("CHAIN_SPACE_UP_PART_STRUCT", "结构件与连接件", "CHAIN_SPACE_UP_PART", 4),
|
||||
("CHAIN_SPACE_UP_PART_THERMAL", "热控与环境控制", "CHAIN_SPACE_UP_PART", 4),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC", "宇航电子", "CHAIN_SPACE_UP", 3),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC_CHIP", "宇航级芯片与器件", "CHAIN_SPACE_UP_ELECTRONIC", 4),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC_CTRL", "制导导航与控制", "CHAIN_SPACE_UP_ELECTRONIC", 4),
|
||||
("CHAIN_SPACE_UP_POWER", "空间电源系统", "CHAIN_SPACE_UP", 3),
|
||||
("CHAIN_SPACE_UP_TEST", "测试与测控设备", "CHAIN_SPACE_UP", 3),
|
||||
("CHAIN_SPACE_MID", "中游研制与发射", "CHAIN_SPACE", 2),
|
||||
("CHAIN_SPACE_MID_SATELLITE", "卫星制造", "CHAIN_SPACE_MID", 3),
|
||||
("CHAIN_SPACE_MID_SAT_PLATFORM", "卫星平台", "CHAIN_SPACE_MID_SATELLITE", 4),
|
||||
("CHAIN_SPACE_MID_SAT_PAYLOAD", "卫星载荷", "CHAIN_SPACE_MID_SATELLITE", 4),
|
||||
("CHAIN_SPACE_MID_SAT_ASSEMBLY", "整星总装与批量制造", "CHAIN_SPACE_MID_SATELLITE", 4),
|
||||
("CHAIN_SPACE_MID_ROCKET", "火箭制造", "CHAIN_SPACE_MID", 3),
|
||||
("CHAIN_SPACE_MID_ROCKET_ENGINE", "火箭发动机与推进", "CHAIN_SPACE_MID_ROCKET", 4),
|
||||
("CHAIN_SPACE_MID_ROCKET_BODY", "箭体结构与分离系统", "CHAIN_SPACE_MID_ROCKET", 4),
|
||||
("CHAIN_SPACE_MID_ROCKET_AVIONICS", "火箭航电与测控", "CHAIN_SPACE_MID_ROCKET", 4),
|
||||
("CHAIN_SPACE_MID_LAUNCH", "发射服务", "CHAIN_SPACE_MID", 3),
|
||||
("CHAIN_SPACE_MID_GROUND", "地面站与终端", "CHAIN_SPACE_MID", 3),
|
||||
("CHAIN_SPACE_MID_GROUND_STATION", "测运控与地面站", "CHAIN_SPACE_MID_GROUND", 4),
|
||||
("CHAIN_SPACE_MID_GROUND_TERMINAL", "卫星终端与天线", "CHAIN_SPACE_MID_GROUND", 4),
|
||||
("CHAIN_SPACE_DOWN", "下游运营与应用", "CHAIN_SPACE", 2),
|
||||
("CHAIN_SPACE_DOWN_COMM", "卫星通信/互联网", "CHAIN_SPACE_DOWN", 3),
|
||||
("CHAIN_SPACE_DOWN_NAV", "导航与遥感", "CHAIN_SPACE_DOWN", 3),
|
||||
("CHAIN_SPACE_DOWN_DATA", "卫星数据服务", "CHAIN_SPACE_DOWN", 3),
|
||||
("CHAIN_SPACE_DOWN_INDUSTRY", "行业应用", "CHAIN_SPACE_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
SPACE_RULES = (
|
||||
("CHAIN_SPACE_UP_MATERIAL", ("航空航天材料", "航天材料", "高温合金", "钛合金", "航天复合材料", "碳纤维复合材料")),
|
||||
("CHAIN_SPACE_UP_PART_STRUCT", ("航天紧固件", "航天连接器", "星箭结构件", "卫星结构件", "火箭结构件")),
|
||||
("CHAIN_SPACE_UP_PART_THERMAL", ("航天热控", "卫星热控", "空间环境控制")),
|
||||
("CHAIN_SPACE_UP_PART", ("航空航天零部件", "航天器零部件", "火箭零部件", "卫星零部件")),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC_CHIP", ("宇航级芯片", "星载电子", "卫星芯片", "宇航电子器件")),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC_CTRL", ("惯性器件", "惯性导航", "制导控制", "姿轨控")),
|
||||
("CHAIN_SPACE_UP_ELECTRONIC", ("航天电子", "宇航电子")),
|
||||
("CHAIN_SPACE_UP_POWER", ("卫星电源系统", "空间能源系统", "卫星能源系统", "空间飞行器能源", "太阳帆板")),
|
||||
("CHAIN_SPACE_UP_TEST", ("航天测控设备", "卫星测控", "航天测试设备", "火箭测试")),
|
||||
("CHAIN_SPACE_MID_SAT_PAYLOAD", ("卫星载荷", "通信载荷", "遥感载荷", "导航载荷")),
|
||||
("CHAIN_SPACE_MID_SAT_PLATFORM", ("卫星平台", "卫星平台舱", "卫星公用平台")),
|
||||
("CHAIN_SPACE_MID_SAT_ASSEMBLY", ("卫星整星", "整星总装", "微小卫星", "卫星批量制造", "卫星及相关产品", "宇航制造")),
|
||||
("CHAIN_SPACE_MID_SATELLITE", ("卫星制造", "卫星研制", "卫星产品")),
|
||||
("CHAIN_SPACE_MID_ROCKET_ENGINE", ("火箭发动机", "液体火箭发动机", "固体火箭发动机", "商业火箭动力", "推进剂")),
|
||||
("CHAIN_SPACE_MID_ROCKET_BODY", ("火箭箭体", "箭体结构", "整流罩", "级间分离")),
|
||||
("CHAIN_SPACE_MID_ROCKET_AVIONICS", ("火箭航电", "火箭测控", "火箭制导")),
|
||||
("CHAIN_SPACE_MID_ROCKET", ("运载火箭", "商业火箭", "火箭研制")),
|
||||
("CHAIN_SPACE_MID_LAUNCH", ("卫星发射", "商业发射", "发射服务", "发射场")),
|
||||
("CHAIN_SPACE_MID_GROUND_STATION", ("卫星地面站", "卫星测运控", "测控站", "地面测控系统")),
|
||||
("CHAIN_SPACE_MID_GROUND_TERMINAL", ("卫星终端", "卫星天线", "地面终端", "卫星通信终端", "相控阵终端")),
|
||||
("CHAIN_SPACE_DOWN_COMM", ("卫星通信", "卫星互联网", "低轨卫星通信", "卫星运营")),
|
||||
("CHAIN_SPACE_DOWN_NAV", ("卫星导航", "北斗导航", "卫星遥感", "遥感应用")),
|
||||
("CHAIN_SPACE_DOWN_DATA", ("卫星数据", "遥感数据", "地理信息服务", "空间信息服务")),
|
||||
("CHAIN_SPACE_DOWN_INDUSTRY", ("卫星行业应用", "卫星应急通信", "卫星海洋应用", "卫星农业应用")),
|
||||
)
|
||||
|
||||
|
||||
CPO_DEFINITIONS = (
|
||||
("CHAIN_CPO", "CPO光通信产业链", "", 1),
|
||||
("CHAIN_CPO_UP", "上游芯片、材料与器件", "CHAIN_CPO", 2),
|
||||
("CHAIN_CPO_UP_CHIP", "光芯片", "CHAIN_CPO_UP", 3),
|
||||
("CHAIN_CPO_UP_CHIP_LASER", "激光器芯片", "CHAIN_CPO_UP_CHIP", 4),
|
||||
("CHAIN_CPO_UP_CHIP_DETECTOR", "探测器芯片", "CHAIN_CPO_UP_CHIP", 4),
|
||||
("CHAIN_CPO_UP_CHIP_SILICON", "硅光芯片", "CHAIN_CPO_UP_CHIP", 4),
|
||||
("CHAIN_CPO_UP_ELECTRIC", "光模块电芯片", "CHAIN_CPO_UP", 3),
|
||||
("CHAIN_CPO_UP_ELECTRIC_DSP", "O-DSP", "CHAIN_CPO_UP_ELECTRIC", 4),
|
||||
("CHAIN_CPO_UP_ELECTRIC_ANALOG", "TIA/Driver", "CHAIN_CPO_UP_ELECTRIC", 4),
|
||||
("CHAIN_CPO_UP_ELECTRIC_SERDES", "SerDes/CDR", "CHAIN_CPO_UP_ELECTRIC", 4),
|
||||
("CHAIN_CPO_UP_COMPONENT", "无源光器件与连接", "CHAIN_CPO_UP", 3),
|
||||
("CHAIN_CPO_UP_COMPONENT_PASSIVE", "FAU/AWG/透镜等无源器件", "CHAIN_CPO_UP_COMPONENT", 4),
|
||||
("CHAIN_CPO_UP_COMPONENT_CONNECT", "MPO与高速连接", "CHAIN_CPO_UP_COMPONENT", 4),
|
||||
("CHAIN_CPO_UP_FIBER", "光纤光缆与材料", "CHAIN_CPO_UP", 3),
|
||||
("CHAIN_CPO_UP_PCB", "高速PCB与基板", "CHAIN_CPO_UP", 3),
|
||||
("CHAIN_CPO_MID", "中游模块与设备", "CHAIN_CPO", 2),
|
||||
("CHAIN_CPO_MID_MODULE", "光模块", "CHAIN_CPO_MID", 3),
|
||||
("CHAIN_CPO_MID_MODULE_PLUG", "可插拔高速光模块", "CHAIN_CPO_MID_MODULE", 4),
|
||||
("CHAIN_CPO_MID_MODULE_LPO", "LPO/LRO", "CHAIN_CPO_MID_MODULE", 4),
|
||||
("CHAIN_CPO_MID_ENGINE", "CPO/光引擎", "CHAIN_CPO_MID", 3),
|
||||
("CHAIN_CPO_MID_ENGINE_SILICON", "硅光光引擎", "CHAIN_CPO_MID_ENGINE", 4),
|
||||
("CHAIN_CPO_MID_ENGINE_PACKAGE", "光电共封装与耦合", "CHAIN_CPO_MID_ENGINE", 4),
|
||||
("CHAIN_CPO_MID_EQUIPMENT", "光通信设备", "CHAIN_CPO_MID", 3),
|
||||
("CHAIN_CPO_MID_SWITCH", "交换机与光互联设备", "CHAIN_CPO_MID", 3),
|
||||
("CHAIN_CPO_DOWN", "下游算力与网络", "CHAIN_CPO", 2),
|
||||
("CHAIN_CPO_DOWN_DC", "AI数据中心", "CHAIN_CPO_DOWN", 3),
|
||||
("CHAIN_CPO_DOWN_TELECOM", "电信网络", "CHAIN_CPO_DOWN", 3),
|
||||
("CHAIN_CPO_DOWN_CLOUD", "云与网络服务", "CHAIN_CPO_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
CPO_RULES = (
|
||||
("CHAIN_CPO_UP_CHIP_LASER", ("激光器芯片", "vcsel", "eml芯片", "dfb芯片", "cw激光器")),
|
||||
("CHAIN_CPO_UP_CHIP_DETECTOR", ("探测器芯片", "光探测器", "pd芯片", "apd芯片")),
|
||||
("CHAIN_CPO_UP_CHIP_SILICON", ("硅光芯片", "硅光子芯片", "silicon photonics")),
|
||||
("CHAIN_CPO_UP_CHIP", ("光芯片",)),
|
||||
("CHAIN_CPO_UP_ELECTRIC_DSP", ("o-dsp", "pam4 dsp", "光模块dsp", "dsp芯片")),
|
||||
("CHAIN_CPO_UP_ELECTRIC_ANALOG", ("tia芯片", "transimpedance amplifier", "laser driver", "driver芯片", "跨阻放大器")),
|
||||
("CHAIN_CPO_UP_ELECTRIC_SERDES", ("serdes", "cdr芯片", "retimer")),
|
||||
("CHAIN_CPO_UP_ELECTRIC", ("光模块电芯片", "光通信电芯片", "光收发电芯片")),
|
||||
("CHAIN_CPO_UP_COMPONENT_PASSIVE", ("fau", "awg", "光隔离器", "光耦合器", "光学透镜", "无源光器件")),
|
||||
("CHAIN_CPO_UP_COMPONENT_CONNECT", ("mpo", "光连接器", "高速连接器", "光纤连接器")),
|
||||
("CHAIN_CPO_UP_COMPONENT", ("光器件", "光学元件")),
|
||||
("CHAIN_CPO_UP_FIBER", ("光纤光缆", "光纤预制棒", "特种光纤", "光通信材料")),
|
||||
("CHAIN_CPO_UP_PCB", ("高速pcb", "高速印制电路板", "高多层印制电路板", "通信高端pcb", "印制电路板的研发、生产和销售", "印制电路板", "电子级玻纤布", "电子级玻璃纤维布", "电子级玻璃纤维超细纱", "高端电子布", "低介电电子布", "超薄电子布")),
|
||||
("CHAIN_CPO_MID_ENGINE_SILICON", ("硅光光引擎", "硅光模块", "硅光引擎")),
|
||||
("CHAIN_CPO_MID_ENGINE_PACKAGE", ("共封装光学", "光电共封装", "cpo封装", "光电耦合")),
|
||||
("CHAIN_CPO_MID_ENGINE", ("cpo光引擎", "光引擎")),
|
||||
("CHAIN_CPO_MID_MODULE_LPO", ("lpo光模块", "lro光模块", "线性驱动光模块")),
|
||||
("CHAIN_CPO_MID_MODULE_PLUG", ("800g光模块", "1.6t光模块", "400g光模块", "可插拔光模块", "高速光模块")),
|
||||
("CHAIN_CPO_MID_MODULE", ("光模块", "光收发模块", "光通信模块", "光收发器")),
|
||||
("CHAIN_CPO_MID_EQUIPMENT", ("光通信设备", "光传输设备", "光网络设备", "光通信系统")),
|
||||
("CHAIN_CPO_MID_SWITCH", ("cpo交换机", "光交换机", "光互联设备")),
|
||||
("CHAIN_CPO_DOWN_DC", ("数据中心光互联", "智算中心光互联", "光网络数据中心")),
|
||||
("CHAIN_CPO_DOWN_TELECOM", ("运营商光网络", "光传输承载网", "5g光承载网")),
|
||||
("CHAIN_CPO_DOWN_CLOUD", ("云数据中心光网络", "云计算光互联")),
|
||||
)
|
||||
|
||||
|
||||
LOW_ALTITUDE_DEFINITIONS = (
|
||||
("CHAIN_LA", "低空经济产业链", "", 1),
|
||||
("CHAIN_LA_UP", "上游材料与核心部件", "CHAIN_LA", 2),
|
||||
("CHAIN_LA_UP_MATERIAL", "航空材料与结构件", "CHAIN_LA_UP", 3),
|
||||
("CHAIN_LA_UP_POWER", "航空电池与动力系统", "CHAIN_LA_UP", 3),
|
||||
("CHAIN_LA_UP_AVIONICS", "航电/飞控/导航", "CHAIN_LA_UP", 3),
|
||||
("CHAIN_LA_UP_SENSOR", "感知与通信设备", "CHAIN_LA_UP", 3),
|
||||
("CHAIN_LA_MID", "低空飞行器制造", "CHAIN_LA", 2),
|
||||
("CHAIN_LA_MID_UAV", "无人机", "CHAIN_LA_MID", 3),
|
||||
("CHAIN_LA_MID_EVTOL", "eVTOL与飞行汽车", "CHAIN_LA_MID", 3),
|
||||
("CHAIN_LA_MID_GA", "通航飞机与直升机", "CHAIN_LA_MID", 3),
|
||||
("CHAIN_LA_INFRA", "基础设施与信息服务", "CHAIN_LA", 2),
|
||||
("CHAIN_LA_INFRA_PHYSICAL", "起降场/无人机机场/能源设施", "CHAIN_LA_INFRA", 3),
|
||||
("CHAIN_LA_INFRA_NETWORK", "低空通信导航监视网络", "CHAIN_LA_INFRA", 3),
|
||||
("CHAIN_LA_INFRA_CONTROL", "空域管理与飞行服务平台", "CHAIN_LA_INFRA", 3),
|
||||
("CHAIN_LA_DOWN", "运营与场景应用", "CHAIN_LA", 2),
|
||||
("CHAIN_LA_DOWN_TRANSPORT", "载人交通与通航运营", "CHAIN_LA_DOWN", 3),
|
||||
("CHAIN_LA_DOWN_LOGISTICS", "低空物流", "CHAIN_LA_DOWN", 3),
|
||||
("CHAIN_LA_DOWN_GOV", "城市治理与应急救援", "CHAIN_LA_DOWN", 3),
|
||||
("CHAIN_LA_DOWN_AGRI", "农林植保", "CHAIN_LA_DOWN", 3),
|
||||
("CHAIN_LA_DOWN_INSPECT", "巡检测绘", "CHAIN_LA_DOWN", 3),
|
||||
("CHAIN_LA_DOWN_TOUR", "文旅与培训", "CHAIN_LA_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
LOW_ALTITUDE_RULES = (
|
||||
("CHAIN_LA_UP_MATERIAL", ("航空复合材料", "航空结构件", "eVTOL结构件", "无人机结构件", "航空铝合金", "航空碳纤维")),
|
||||
("CHAIN_LA_UP_POWER", ("航空电池", "eVTOL电池", "航空电机", "航空电驱", "电动航空动力系统", "航空发动机")),
|
||||
("CHAIN_LA_UP_AVIONICS", ("航空电子", "飞行控制系统", "无人机飞控", "航空导航", "航电系统")),
|
||||
("CHAIN_LA_UP_SENSOR", ("低空雷达", "无人机通信", "低空通信", "机载通信", "机载传感器", "反无人机设备")),
|
||||
("CHAIN_LA_MID_EVTOL", ("evtol整机", "电动垂直起降飞行器", "飞行汽车整机", "载人无人驾驶航空器整机")),
|
||||
("CHAIN_LA_MID_UAV", ("无人机整机", "工业无人机", "民用无人机", "物流无人机", "植保无人机")),
|
||||
("CHAIN_LA_MID_GA", ("通用航空器", "通航飞机", "轻型飞机", "直升机制造")),
|
||||
("CHAIN_LA_INFRA_PHYSICAL", ("低空起降场", "垂直起降场", "无人机机场", "低空充换电", "低空能源站")),
|
||||
("CHAIN_LA_INFRA_NETWORK", ("低空通信导航监视", "低空智联网", "低空通信网", "低空监视网", "低空导航网")),
|
||||
("CHAIN_LA_INFRA_CONTROL", ("低空飞行服务平台", "低空空域管理", "无人机交通管理", "uom平台", "低空航路规划")),
|
||||
("CHAIN_LA_DOWN_TRANSPORT", ("低空载人交通", "空中交通运营", "通用航空运营", "短途运输航空")),
|
||||
("CHAIN_LA_DOWN_LOGISTICS", ("低空物流", "无人机配送", "无人机物流运营")),
|
||||
("CHAIN_LA_DOWN_GOV", ("无人机应急救援", "低空城市治理", "无人机消防", "无人机警务")),
|
||||
("CHAIN_LA_DOWN_AGRI", ("无人机植保服务", "农林植保服务", "农业无人机运营")),
|
||||
("CHAIN_LA_DOWN_INSPECT", ("无人机巡检", "无人机测绘", "低空遥感", "电力巡检无人机")),
|
||||
("CHAIN_LA_DOWN_TOUR", ("低空旅游", "飞行培训", "无人机培训", "航空运动")),
|
||||
)
|
||||
|
||||
|
||||
AIDC_DEFINITIONS = (
|
||||
("CHAIN_AIDC", "AI算力与数据中心产业链", "", 1),
|
||||
("CHAIN_AIDC_UP", "上游算力硬件", "CHAIN_AIDC", 2),
|
||||
("CHAIN_AIDC_UP_CHIP", "AI芯片与加速卡", "CHAIN_AIDC_UP", 3),
|
||||
("CHAIN_AIDC_UP_SERVER", "AI服务器与整机柜", "CHAIN_AIDC_UP", 3),
|
||||
("CHAIN_AIDC_UP_STORAGE", "存储与高速互联", "CHAIN_AIDC_UP", 3),
|
||||
("CHAIN_AIDC_UP_NETWORK", "云计算与网络设备", "CHAIN_AIDC_UP", 3),
|
||||
("CHAIN_AIDC_INFRA", "数据中心基础设施", "CHAIN_AIDC", 2),
|
||||
("CHAIN_AIDC_INFRA_POWER", "供配电系统", "CHAIN_AIDC_INFRA", 3),
|
||||
("CHAIN_AIDC_INFRA_POWER_HVDC", "UPS/HVDC/SST/PDU", "CHAIN_AIDC_INFRA_POWER", 4),
|
||||
("CHAIN_AIDC_INFRA_GRID", "变压器与电网接入", "CHAIN_AIDC_INFRA", 3),
|
||||
("CHAIN_AIDC_INFRA_COOL", "液冷与热管理", "CHAIN_AIDC_INFRA", 3),
|
||||
("CHAIN_AIDC_INFRA_COOL_COLD", "冷板液冷/CDU", "CHAIN_AIDC_INFRA_COOL", 4),
|
||||
("CHAIN_AIDC_INFRA_COOL_IMMERSE", "浸没式液冷", "CHAIN_AIDC_INFRA_COOL", 4),
|
||||
("CHAIN_AIDC_INFRA_ENERGY", "备用与现场能源", "CHAIN_AIDC_INFRA", 3),
|
||||
("CHAIN_AIDC_INFRA_BUILD", "机房建设与配套", "CHAIN_AIDC_INFRA", 3),
|
||||
("CHAIN_AIDC_MID", "算力中心运营与服务", "CHAIN_AIDC", 2),
|
||||
("CHAIN_AIDC_MID_IDC", "IDC与智算中心运营", "CHAIN_AIDC_MID", 3),
|
||||
("CHAIN_AIDC_MID_RENT", "算力租赁与调度", "CHAIN_AIDC_MID", 3),
|
||||
("CHAIN_AIDC_MID_CLOUD", "云计算与模型服务", "CHAIN_AIDC_MID", 3),
|
||||
("CHAIN_AIDC_DOWN", "下游AI应用", "CHAIN_AIDC", 2),
|
||||
("CHAIN_AIDC_DOWN_ENTERPRISE", "企业与行业AI", "CHAIN_AIDC_DOWN", 3),
|
||||
("CHAIN_AIDC_DOWN_EDGE", "边缘与端侧AI", "CHAIN_AIDC_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
AIDC_RULES = (
|
||||
("CHAIN_AIDC_UP_CHIP", ("ai芯片", "gpu加速卡", "ai加速卡", "算力芯片", "训练芯片", "推理芯片", "数据中心asic")),
|
||||
("CHAIN_AIDC_UP_SERVER", ("ai服务器", "ai服务器业务", "智算服务器", "gpu服务器", "超节点", "整机柜服务器")),
|
||||
("CHAIN_AIDC_UP_STORAGE", ("ai存储", "全闪存储", "高速互联", "数据中心互联", "dci设备", "智能网卡")),
|
||||
("CHAIN_AIDC_UP_NETWORK", ("通信及移动网络设备、云计算", "云计算设备", "云网络设备", "数据中心网络设备", "网络通信设备")),
|
||||
("CHAIN_AIDC_INFRA_POWER_HVDC", ("数据中心ups", "数据中心hvdc", "800v直流", "固态变压器", "数据中心pdu", "服务器电源")),
|
||||
("CHAIN_AIDC_INFRA_POWER", ("数据中心电源", "数据中心供配电", "智算中心供配电")),
|
||||
("CHAIN_AIDC_INFRA_GRID", ("数据中心变压器", "智算中心变压器", "数据中心电网接入", "数据中心开关柜")),
|
||||
("CHAIN_AIDC_INFRA_COOL_COLD", ("冷板液冷", "液冷冷板", "液冷cdu", "液冷分配单元")),
|
||||
("CHAIN_AIDC_INFRA_COOL_IMMERSE", ("浸没式液冷", "单相浸没液冷", "两相浸没液冷")),
|
||||
("CHAIN_AIDC_INFRA_COOL", ("数据中心液冷", "服务器液冷", "智算中心液冷", "数据中心热管理", "数据中心温控", "数据中心温控系统")),
|
||||
("CHAIN_AIDC_INFRA_ENERGY", ("数据中心备用电源", "数据中心柴油发电机", "数据中心燃气轮机", "数据中心sofc", "数据中心储能")),
|
||||
("CHAIN_AIDC_INFRA_BUILD", ("数据中心机房建设", "数据中心基础设施建设", "数据中心工程", "智算中心建设", "数据中心解决方案", "智算中心工程", "液冷数据中心工程")),
|
||||
("CHAIN_AIDC_MID_IDC", ("idc运营", "数据中心运营", "智算中心运营", "互联网数据中心服务")),
|
||||
("CHAIN_AIDC_MID_RENT", ("算力租赁", "算力调度", "算力服务平台", "智算服务")),
|
||||
("CHAIN_AIDC_MID_CLOUD", ("云计算服务", "云计算及存储", "数云服务", "算力云服务", "大模型服务", "模型即服务", "maas平台")),
|
||||
("CHAIN_AIDC_DOWN_ENTERPRISE", ("企业ai应用", "行业大模型", "人工智能解决方案", "ai营销", "ai应用分发", "ai应用平台", "空间智能maas")),
|
||||
("CHAIN_AIDC_DOWN_EDGE", ("边缘ai", "端侧ai", "边缘算力")),
|
||||
)
|
||||
|
||||
|
||||
MILITARY_ELECTRONICS_DEFINITIONS = (
|
||||
("CHAIN_MIL", "军工电子产业链", "", 1),
|
||||
("CHAIN_MIL_UP", "基础材料与元器件", "CHAIN_MIL", 2),
|
||||
("CHAIN_MIL_UP_CHIP", "军用芯片与器件", "CHAIN_MIL_UP", 3),
|
||||
("CHAIN_MIL_UP_PASSIVE", "被动元件与连接器", "CHAIN_MIL_UP", 3),
|
||||
("CHAIN_MIL_UP_POWER", "军用电源与电机", "CHAIN_MIL_UP", 3),
|
||||
("CHAIN_MIL_UP_MATERIAL", "电子材料与工艺", "CHAIN_MIL_UP", 3),
|
||||
("CHAIN_MIL_MID", "探测通信与控制", "CHAIN_MIL", 2),
|
||||
("CHAIN_MIL_MID_RADAR", "雷达与探测", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_MID_COMM", "军用通信与数据链", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_MID_NAV", "导航制导与控制", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_MID_EW", "电子对抗", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_MID_OPTO", "红外与光电系统", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_MID_TEST", "测试测量与仿真", "CHAIN_MIL_MID", 3),
|
||||
("CHAIN_MIL_DOWN", "分系统与信息化", "CHAIN_MIL", 2),
|
||||
("CHAIN_MIL_DOWN_AVIONICS", "航空航电系统", "CHAIN_MIL_DOWN", 3),
|
||||
("CHAIN_MIL_DOWN_SHIP", "舰船电子系统", "CHAIN_MIL_DOWN", 3),
|
||||
("CHAIN_MIL_DOWN_C4ISR", "指挥控制与C4ISR", "CHAIN_MIL_DOWN", 3),
|
||||
("CHAIN_MIL_DOWN_INTEGRATION", "军工电子系统集成", "CHAIN_MIL_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
MILITARY_ELECTRONICS_RULES = (
|
||||
("CHAIN_MIL_UP_CHIP", ("军用芯片", "宇航级芯片", "军用集成电路", "抗辐照芯片", "军用半导体器件")),
|
||||
("CHAIN_MIL_UP_PASSIVE", ("军用连接器", "军用电容器", "军用继电器", "军用电阻器", "军用电子元器件")),
|
||||
("CHAIN_MIL_UP_POWER", ("军用电源", "航空电源", "军用电机", "高可靠电源模块")),
|
||||
("CHAIN_MIL_UP_MATERIAL", ("军用电子材料", "微波介质材料", "军用覆铜板", "电子封装材料")),
|
||||
("CHAIN_MIL_MID_RADAR", ("军用雷达", "相控阵雷达", "雷达整机", "雷达射频组件", "雷达探测系统")),
|
||||
("CHAIN_MIL_MID_COMM", ("军用通信", "军用无线电", "战术数据链", "军用通信设备", "军用卫星通信")),
|
||||
("CHAIN_MIL_MID_NAV", ("惯性导航系统", "军用导航", "导引头", "精确制导", "飞行控制系统")),
|
||||
("CHAIN_MIL_MID_EW", ("电子对抗", "电子侦察", "雷达干扰", "通信对抗")),
|
||||
("CHAIN_MIL_MID_OPTO", ("军用红外", "红外探测器", "军用光电系统", "光电吊舱", "夜视设备")),
|
||||
("CHAIN_MIL_MID_TEST", ("军用测试设备", "军用ATE", "军工仿真", "射频测试系统")),
|
||||
("CHAIN_MIL_DOWN_AVIONICS", ("航空电子系统", "综合航电", "机载电子系统")),
|
||||
("CHAIN_MIL_DOWN_SHIP", ("舰船电子", "舰载电子系统", "水声探测系统")),
|
||||
("CHAIN_MIL_DOWN_C4ISR", ("c4isr", "指挥控制系统", "作战指挥系统", "战场信息系统")),
|
||||
("CHAIN_MIL_DOWN_INTEGRATION", ("军工电子系统集成", "军用信息化系统", "国防信息化系统")),
|
||||
)
|
||||
|
||||
|
||||
CONSUMER_ELECTRONICS_DEFINITIONS = (
|
||||
("CHAIN_CE", "消费电子产业链", "", 1),
|
||||
("CHAIN_CE_UP", "材料与核心器件", "CHAIN_CE", 2),
|
||||
("CHAIN_CE_UP_MATERIAL", "结构与功能材料", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_PCB", "PCB/FPC与连接器", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_CHIP", "端侧芯片与存储", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_DISPLAY", "显示面板与触控", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_OPTICAL", "光学器件", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_ACOUSTIC", "声学与触觉器件", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_UP_POWER", "电池与充电器件", "CHAIN_CE_UP", 3),
|
||||
("CHAIN_CE_MID", "模组与整机制造", "CHAIN_CE", 2),
|
||||
("CHAIN_CE_MID_CAMERA", "摄像头模组", "CHAIN_CE_MID", 3),
|
||||
("CHAIN_CE_MID_DISPLAY", "显示/触控模组", "CHAIN_CE_MID", 3),
|
||||
("CHAIN_CE_MID_ODM", "精密制造与ODM/EMS", "CHAIN_CE_MID", 3),
|
||||
("CHAIN_CE_DOWN", "品牌终端", "CHAIN_CE", 2),
|
||||
("CHAIN_CE_DOWN_PHONE", "智能手机", "CHAIN_CE_DOWN", 3),
|
||||
("CHAIN_CE_DOWN_PC", "PC/平板", "CHAIN_CE_DOWN", 3),
|
||||
("CHAIN_CE_DOWN_WEARABLE", "智能穿戴与TWS", "CHAIN_CE_DOWN", 3),
|
||||
("CHAIN_CE_DOWN_XR", "XR设备", "CHAIN_CE_DOWN", 3),
|
||||
("CHAIN_CE_DOWN_GLASSES", "AI眼镜", "CHAIN_CE_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
CONSUMER_ELECTRONICS_RULES = (
|
||||
("CHAIN_CE_UP_MATERIAL", ("消费电子结构件", "消费电子功能材料", "手机盖板玻璃", "消费电子金属中框", "电磁屏蔽材料")),
|
||||
("CHAIN_CE_UP_PCB", ("消费电子pcb", "消费电子fpc", "手机连接器", "消费电子连接器", "柔性线路板")),
|
||||
("CHAIN_CE_UP_CHIP", ("手机soc", "端侧ai芯片", "消费电子芯片", "手机存储芯片", "智能终端芯片")),
|
||||
("CHAIN_CE_UP_DISPLAY", ("手机显示面板", "消费电子显示屏", "oled面板", "触控面板", "显示器件业务", "液晶显示面板", "tft-lcd", "amoled显示", "mled业务")),
|
||||
("CHAIN_CE_UP_OPTICAL", ("手机镜头", "消费电子光学镜头", "光学棱镜", "微型投影光机")),
|
||||
("CHAIN_CE_UP_ACOUSTIC", ("消费电子声学器件", "微型扬声器", "tws声学", "线性马达", "触觉反馈器件")),
|
||||
("CHAIN_CE_UP_POWER", ("消费电子电池", "手机电池", "tws电池", "消费电子充电器", "无线充电模组")),
|
||||
("CHAIN_CE_MID_CAMERA", ("手机摄像头模组", "消费电子摄像头模组", "3d感知模组")),
|
||||
("CHAIN_CE_MID_DISPLAY", ("显示模组", "触控模组", "oled模组")),
|
||||
("CHAIN_CE_MID_ODM", ("消费电子整机代工", "智能终端odm", "电子制造服务ems", "手机精密组装", "智能终端的结构件、功能模组、整机组装", "整机组装等配套服务")),
|
||||
("CHAIN_CE_DOWN_PHONE", ("智能手机整机", "手机品牌业务")),
|
||||
("CHAIN_CE_DOWN_PC", ("笔记本电脑整机", "平板电脑整机", "ai pc整机")),
|
||||
("CHAIN_CE_DOWN_GLASSES", ("ai眼镜整机", "智能眼镜整机")),
|
||||
("CHAIN_CE_DOWN_XR", ("vr整机", "ar整机", "xr设备", "虚拟现实终端")),
|
||||
("CHAIN_CE_DOWN_WEARABLE", ("智能手表", "智能手环", "tws耳机整机", "可穿戴设备整机")),
|
||||
)
|
||||
|
||||
|
||||
SOFTWARE_CLOUD_DEFINITIONS = (
|
||||
("CHAIN_SW", "软件云与网络安全产业链", "", 1),
|
||||
("CHAIN_SW_UP", "基础软件与工具", "CHAIN_SW", 2),
|
||||
("CHAIN_SW_UP_OS", "操作系统", "CHAIN_SW_UP", 3),
|
||||
("CHAIN_SW_UP_DB", "数据库", "CHAIN_SW_UP", 3),
|
||||
("CHAIN_SW_UP_MIDDLEWARE", "中间件", "CHAIN_SW_UP", 3),
|
||||
("CHAIN_SW_UP_DEV", "开发工具与开源生态", "CHAIN_SW_UP", 3),
|
||||
("CHAIN_SW_MID_CLOUD", "云计算服务", "CHAIN_SW", 2),
|
||||
("CHAIN_SW_MID_IAAS", "IaaS与云基础设施", "CHAIN_SW_MID_CLOUD", 3),
|
||||
("CHAIN_SW_MID_PAAS", "PaaS与云原生", "CHAIN_SW_MID_CLOUD", 3),
|
||||
("CHAIN_SW_MID_SAAS", "SaaS", "CHAIN_SW_MID_CLOUD", 3),
|
||||
("CHAIN_SW_MID_SECURITY", "网络与数据安全", "CHAIN_SW", 2),
|
||||
("CHAIN_SW_MID_SEC_NETWORK", "网络边界与终端安全", "CHAIN_SW_MID_SECURITY", 3),
|
||||
("CHAIN_SW_MID_SEC_DATA", "数据安全与密码", "CHAIN_SW_MID_SECURITY", 3),
|
||||
("CHAIN_SW_MID_SEC_CLOUD", "云安全与身份安全", "CHAIN_SW_MID_SECURITY", 3),
|
||||
("CHAIN_SW_MID_INDUSTRIAL", "工业软件", "CHAIN_SW", 2),
|
||||
("CHAIN_SW_MID_RD", "研发设计软件", "CHAIN_SW_MID_INDUSTRIAL", 3),
|
||||
("CHAIN_SW_MID_PRODUCTION", "生产控制软件", "CHAIN_SW_MID_INDUSTRIAL", 3),
|
||||
("CHAIN_SW_MID_MANAGEMENT", "经营管理软件", "CHAIN_SW_MID_INDUSTRIAL", 3),
|
||||
("CHAIN_SW_DOWN", "行业应用软件", "CHAIN_SW", 2),
|
||||
("CHAIN_SW_DOWN_GOV", "政务与公共服务", "CHAIN_SW_DOWN", 3),
|
||||
("CHAIN_SW_DOWN_FINANCE", "金融科技", "CHAIN_SW_DOWN", 3),
|
||||
("CHAIN_SW_DOWN_MEDICAL", "医疗信息化", "CHAIN_SW_DOWN", 3),
|
||||
("CHAIN_SW_DOWN_ENTERPRISE", "企业数字化", "CHAIN_SW_DOWN", 3),
|
||||
("CHAIN_SW_DOWN_AI", "AI应用与智能体", "CHAIN_SW_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
SOFTWARE_CLOUD_RULES = (
|
||||
("CHAIN_SW_UP_OS", ("国产操作系统", "服务器操作系统", "桌面操作系统", "嵌入式操作系统")),
|
||||
("CHAIN_SW_UP_DB", ("国产数据库", "关系型数据库", "分布式数据库", "数据库管理系统")),
|
||||
("CHAIN_SW_UP_MIDDLEWARE", ("应用中间件", "消息中间件", "交易中间件", "国产中间件")),
|
||||
("CHAIN_SW_UP_DEV", ("软件开发工具", "低代码开发平台", "devops平台", "开源软件平台")),
|
||||
("CHAIN_SW_MID_IAAS", ("iaas云服务", "公有云基础设施", "私有云平台", "云计算基础设施服务")),
|
||||
("CHAIN_SW_MID_PAAS", ("paas平台", "云原生平台", "容器云平台", "微服务平台")),
|
||||
("CHAIN_SW_MID_SAAS", ("saas软件", "企业级saas", "云办公软件", "软件即服务")),
|
||||
("CHAIN_SW_MID_SEC_NETWORK", ("网络安全产品", "防火墙产品", "终端安全", "入侵检测", "安全运营中心")),
|
||||
("CHAIN_SW_MID_SEC_DATA", ("数据安全产品", "数据防泄漏", "商用密码", "密码安全", "隐私计算")),
|
||||
("CHAIN_SW_MID_SEC_CLOUD", ("云安全产品", "身份安全", "零信任安全", "云安全服务")),
|
||||
("CHAIN_SW_MID_RD", ("工业cad", "cae软件", "eda软件", "plm软件", "研发设计类工业软件")),
|
||||
("CHAIN_SW_MID_PRODUCTION", ("mes系统", "scada系统", "工业控制软件", "生产控制类工业软件")),
|
||||
("CHAIN_SW_MID_MANAGEMENT", ("erp软件", "供应链管理软件", "企业管理软件", "经营管理类工业软件")),
|
||||
("CHAIN_SW_DOWN_GOV", ("政务软件", "数字政府平台", "智慧政务系统")),
|
||||
("CHAIN_SW_DOWN_FINANCE", ("银行核心系统", "证券交易软件", "金融科技软件", "保险信息系统")),
|
||||
("CHAIN_SW_DOWN_MEDICAL", ("医院信息系统", "医疗信息化软件", "电子病历系统")),
|
||||
("CHAIN_SW_DOWN_ENTERPRISE", ("企业数字化软件", "协同办公软件", "客户关系管理软件")),
|
||||
("CHAIN_SW_DOWN_AI", ("ai应用软件", "智能体平台", "企业级智能体", "大模型应用软件")),
|
||||
)
|
||||
|
||||
|
||||
ELECTRONIC_COMPONENT_DEFINITIONS = (
|
||||
("CHAIN_EC", "电子元件产业链", "", 1),
|
||||
("CHAIN_EC_UP", "上游材料与设备", "CHAIN_EC", 2),
|
||||
("CHAIN_EC_UP_CCL", "覆铜板与基材", "CHAIN_EC_UP", 3),
|
||||
("CHAIN_EC_UP_COPPER", "电子铜箔", "CHAIN_EC_UP", 3),
|
||||
("CHAIN_EC_UP_EQUIPMENT", "元件与PCB设备", "CHAIN_EC_UP", 3),
|
||||
("CHAIN_EC_PCB", "PCB制造", "CHAIN_EC", 2),
|
||||
("CHAIN_EC_PCB_RIGID", "刚性板与HDI", "CHAIN_EC_PCB", 3),
|
||||
("CHAIN_EC_PCB_FLEX", "柔性线路板", "CHAIN_EC_PCB", 3),
|
||||
("CHAIN_EC_PCB_SUBSTRATE", "IC载板", "CHAIN_EC_PCB", 3),
|
||||
("CHAIN_EC_PASSIVE", "被动元件", "CHAIN_EC", 2),
|
||||
("CHAIN_EC_PASSIVE_CAP", "电容器", "CHAIN_EC_PASSIVE", 3),
|
||||
("CHAIN_EC_PASSIVE_RES", "电阻器", "CHAIN_EC_PASSIVE", 3),
|
||||
("CHAIN_EC_PASSIVE_IND", "电感与磁性元件", "CHAIN_EC_PASSIVE", 3),
|
||||
("CHAIN_EC_PASSIVE_CRYSTAL", "晶振与频率器件", "CHAIN_EC_PASSIVE", 3),
|
||||
("CHAIN_EC_CONNECT", "互连与机电元件", "CHAIN_EC", 2),
|
||||
("CHAIN_EC_CONNECTOR", "连接器", "CHAIN_EC_CONNECT", 3),
|
||||
("CHAIN_EC_RELAY", "继电器与开关", "CHAIN_EC_CONNECT", 3),
|
||||
("CHAIN_EC_RF", "射频与微波元件", "CHAIN_EC_CONNECT", 3),
|
||||
("CHAIN_EC_SENSOR", "传感器与敏感元件", "CHAIN_EC_CONNECT", 3),
|
||||
)
|
||||
|
||||
|
||||
ELECTRONIC_COMPONENT_RULES = (
|
||||
("CHAIN_EC_UP_CCL", ("覆铜板生产", "覆铜板的研发、生产和销售", "电子行业基础材料覆铜板", "高频高速覆铜板", "铜箔基板", "电子级玻纤布", "电子级玻璃纤维布", "电子级玻璃纤维超细纱", "半固化片")),
|
||||
("CHAIN_EC_UP_COPPER", ("电子铜箔生产", "电子电路铜箔", "pcb铜箔", "覆铜板用铜箔", "高频高速铜箔")),
|
||||
("CHAIN_EC_UP_EQUIPMENT", ("pcb专用设备", "线路板设备", "pcb钻孔设备", "pcb检测设备", "电子元件专用设备", "为pcb、数控精密机件等领域", "pcb工具、材料、装备")),
|
||||
("CHAIN_EC_PCB_SUBSTRATE", ("ic载板", "封装载板", "集成电路载板")),
|
||||
("CHAIN_EC_PCB_FLEX", ("柔性线路板", "柔性印制电路板", "fpc产品", "软硬结合板")),
|
||||
("CHAIN_EC_PCB_RIGID", ("印制电路板生产", "印刷电路板生产", "印制电路板", "刚性线路板", "高密度互连板", "hdi板", "多层线路板")),
|
||||
("CHAIN_EC_PASSIVE_CAP", ("电容器生产", "铝电解电容器", "薄膜电容器", "陶瓷电容器", "mlcc产品", "钽电容")),
|
||||
("CHAIN_EC_PASSIVE_RES", ("电阻器生产", "片式电阻", "厚膜电阻", "精密电阻")),
|
||||
("CHAIN_EC_PASSIVE_IND", ("电感器生产", "片式电感", "功率电感", "磁性元件", "电子变压器")),
|
||||
("CHAIN_EC_PASSIVE_CRYSTAL", ("石英晶体元器件", "晶体谐振器", "晶体振荡器", "晶振产品")),
|
||||
("CHAIN_EC_CONNECTOR", ("电子连接器", "通信连接器", "汽车连接器", "高速连接器", "连接器产品")),
|
||||
("CHAIN_EC_RELAY", ("继电器生产", "电磁继电器", "固态继电器", "微动开关", "电子开关")),
|
||||
("CHAIN_EC_RF", ("射频元器件", "微波元器件", "滤波器产品", "双工器", "射频前端器件")),
|
||||
("CHAIN_EC_SENSOR", ("传感器产品", "敏感元器件", "温度传感器", "压力传感器", "气体传感器")),
|
||||
)
|
||||
|
||||
|
||||
COMMUNICATION_NETWORK_DEFINITIONS = (
|
||||
("CHAIN_COMM", "通信网络与服务产业链", "", 1),
|
||||
("CHAIN_COMM_UP", "上游芯片器件与线缆", "CHAIN_COMM", 2),
|
||||
("CHAIN_COMM_UP_CHIP", "通信芯片与模组", "CHAIN_COMM_UP", 3),
|
||||
("CHAIN_COMM_UP_RF", "射频器件与天线", "CHAIN_COMM_UP", 3),
|
||||
("CHAIN_COMM_UP_CABLE", "光纤光缆与通信线缆", "CHAIN_COMM_UP", 3),
|
||||
("CHAIN_COMM_MID", "中游网络设备", "CHAIN_COMM", 2),
|
||||
("CHAIN_COMM_MID_BASE", "基站与无线接入设备", "CHAIN_COMM_MID", 3),
|
||||
("CHAIN_COMM_MID_TRANSPORT", "传输与承载设备", "CHAIN_COMM_MID", 3),
|
||||
("CHAIN_COMM_MID_SWITCH", "交换机与路由器", "CHAIN_COMM_MID", 3),
|
||||
("CHAIN_COMM_MID_ACCESS", "宽带接入与终端", "CHAIN_COMM_MID", 3),
|
||||
("CHAIN_COMM_MID_SAT", "卫星通信设备与终端", "CHAIN_COMM_MID", 3),
|
||||
("CHAIN_COMM_INFRA", "网络基础设施", "CHAIN_COMM", 2),
|
||||
("CHAIN_COMM_INFRA_TOWER", "通信塔与机房设施", "CHAIN_COMM_INFRA", 3),
|
||||
("CHAIN_COMM_INFRA_PLAN", "网络规划设计", "CHAIN_COMM_INFRA", 3),
|
||||
("CHAIN_COMM_INFRA_BUILD", "网络建设与运维", "CHAIN_COMM_INFRA", 3),
|
||||
("CHAIN_COMM_DOWN", "下游运营与通信服务", "CHAIN_COMM", 2),
|
||||
("CHAIN_COMM_DOWN_OPERATOR", "基础电信运营", "CHAIN_COMM_DOWN", 3),
|
||||
("CHAIN_COMM_DOWN_BROADBAND", "宽带与专线服务", "CHAIN_COMM_DOWN", 3),
|
||||
("CHAIN_COMM_DOWN_IOT", "物联网连接服务", "CHAIN_COMM_DOWN", 3),
|
||||
("CHAIN_COMM_DOWN_ENTERPRISE", "企业通信与云网服务", "CHAIN_COMM_DOWN", 3),
|
||||
)
|
||||
|
||||
|
||||
COMMUNICATION_NETWORK_RULES = (
|
||||
("CHAIN_COMM_UP_CHIP", ("通信基带芯片", "射频通信芯片", "蜂窝通信模组", "无线通信模组", "物联网通信模组")),
|
||||
("CHAIN_COMM_UP_RF", ("基站射频器件", "基站天线", "射频滤波器", "通信天线", "天线振子")),
|
||||
("CHAIN_COMM_UP_CABLE", ("光纤光缆生产", "通信光缆", "通信电缆", "海底通信光缆")),
|
||||
("CHAIN_COMM_MID_BASE", ("移动通信基站", "基站设备", "无线接入网设备", "小基站产品")),
|
||||
("CHAIN_COMM_MID_TRANSPORT", ("光传输设备", "通信传输设备", "分组传送网设备", "承载网设备")),
|
||||
("CHAIN_COMM_MID_SWITCH", ("数据中心交换机", "以太网交换机", "核心路由器", "企业级路由器")),
|
||||
("CHAIN_COMM_MID_ACCESS", ("宽带接入设备", "光网络终端", "家庭网关", "通信终端设备", "cpe终端")),
|
||||
("CHAIN_COMM_MID_SAT", ("卫星通信终端", "卫星通信设备", "卫星通信系统", "卫星互联网终端")),
|
||||
("CHAIN_COMM_INFRA_TOWER", ("通信铁塔", "通信塔产品", "通信机房", "基站配套设施")),
|
||||
("CHAIN_COMM_INFRA_PLAN", ("通信网络规划设计", "通信工程勘察设计", "无线网络优化设计")),
|
||||
("CHAIN_COMM_INFRA_BUILD", ("通信工程建设", "通信网络维护", "通信网络优化", "通信设施运维")),
|
||||
("CHAIN_COMM_DOWN_OPERATOR", ("移动通信运营", "基础电信业务", "电信运营服务", "移动通信服务")),
|
||||
("CHAIN_COMM_DOWN_BROADBAND", ("固定宽带业务", "互联网专线服务", "企业专线业务", "宽带接入服务")),
|
||||
("CHAIN_COMM_DOWN_IOT", ("物联网连接服务", "物联网卡业务", "蜂窝物联网运营")),
|
||||
("CHAIN_COMM_DOWN_ENTERPRISE", ("企业通信服务", "云网融合服务", "统一通信服务", "通信云服务")),
|
||||
)
|
||||
@@ -0,0 +1,191 @@
|
||||
"""Score business evidence against product rules without I/O."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
|
||||
from .models import ChainSpec
|
||||
from .catalogs import CHAIN_SPECS
|
||||
from .catalogs import (
|
||||
COAL_CHEMICAL_RULES, INFRASTRUCTURE_RULES, ELECTRONIC_COMPONENT_RULES,
|
||||
COMMUNICATION_NETWORK_RULES, AUTO_AFTERMARKET_RULES, REAL_ESTATE_RULES,
|
||||
BUILDING_MATERIAL_RULES, STEEL_RULES, NONFERROUS_RULES, FINANCE_RULES,
|
||||
BUSINESS_INDUSTRY_RULES,
|
||||
)
|
||||
from .evidence import _profile_texts, _split_f10_profile_evidence
|
||||
|
||||
_WEAK_BUSINESS_CLAUSE_MARKERS = (
|
||||
"尚无", "暂无", "不涉及", "不生产", "未开展", "未形成", "拟开展", "拟布局", "计划",
|
||||
"择机", "关注", "探索", "目标客户", "收入规模较小", "不构成重大影响", "社会责任",
|
||||
"帮扶", "捐资", "捐赠", "设有",
|
||||
)
|
||||
|
||||
|
||||
_DOWNSTREAM_CONTEXT_MARKERS = (
|
||||
"用于", "应用于", "适用于", "可用于", "运用于", "参股", "持有", "投资", "供应商",
|
||||
)
|
||||
|
||||
|
||||
_EQUIPMENT_SUFFIX_MARKERS = (
|
||||
"生产线", "线", "用", "设备", "装备", "机械", "部件", "零部件", "组件", "配线", "线束", "控制器", "转鼓",
|
||||
)
|
||||
|
||||
|
||||
_PROFILE_ONLY_KEYWORDS = {
|
||||
"教育信息化", "智慧教育", "智慧教育平台", "数字化人才培养", "职业教育业务", "课程研发",
|
||||
"空调压缩机", "冰箱压缩机", "家电电机", "智能家居业务", "智能家居系统", "电视产品", "电视整机",
|
||||
"智能电视整机", "激光电视整机", "液晶电视生产", "小家电产品", "生活电器产品", "清洁电器产品",
|
||||
"厨房电器产品", "厨卫电器产品", "制冷配件", "空调阀件", "电视面板",
|
||||
"小家电", "厨房小家电", "生活小家电", "个护小家电", "智慧显示终端", "彩电业务", "白电业务",
|
||||
"小型家电", "家电控制器", "家用电器控制器", "生活电器控制器", "家电智能控制器",
|
||||
"定制家居", "全屋定制", "软体家具", "沙发", "床垫", "纸包装", "金属包装", "生活用纸", "家居五金", "智能马桶",
|
||||
"瓦楞包装", "彩盒包装", "纸质包装", "纸制品包装", "三片罐", "二片罐", "金属易拉罐",
|
||||
"纱线", "棉纱", "色纱", "面料", "坯布", "印染", "成衣", "服装生产", "服装制造", "家纺产品",
|
||||
"家用纺织品", "床上用品", "家纺业务", "鞋履", "皮鞋", "男装", "童装", "内衣", "户外用品", "运动鞋",
|
||||
"服装智能制造", "职业装", "校服", "服饰辅料", "服装辅料", "拉链", "钮扣", "纽扣",
|
||||
"非织造材料", "非织造布", "无纺布",
|
||||
"ip版权运营", "内容版权运营", "版权代理", "版权管理", "版权交易", "数字内容技术",
|
||||
"虚拟拍摄", "影视后期制作", "影视特效制作", "图书出版", "教材出版", "期刊出版", "数字出版",
|
||||
"出版发行业务", "出版物编辑", "电影制作", "电影投资制作", "电影发行", "电影制片", "电视剧制作",
|
||||
"影视剧制作", "影视内容制作", "综艺节目制作", "电视节目制作", "动画制作", "动漫内容", "动漫影视",
|
||||
"动画电影", "短剧制作", "短剧业务", "漫剧制作", "互动短剧", "小游戏研发", "微信小游戏", "小程序游戏",
|
||||
"休闲小游戏", "移动游戏研发", "网络游戏研发", "客户端游戏", "手机游戏", "主机游戏", "广告创意",
|
||||
"广告内容制作", "影院运营", "电影院线", "电影放映", "影城运营", "广播电视节目播出", "有线电视网络",
|
||||
"广播电视网络", "电视台运营", "长视频平台", "在线视频平台", "视频平台运营", "互联网视频服务",
|
||||
"短视频平台", "mcn业务", "达人经纪", "直播经纪", "游戏发行运营", "网络游戏运营", "移动游戏运营",
|
||||
"游戏运营业务", "广告代理", "数字营销", "整合营销", "效果营销", "媒介代理", "营销服务",
|
||||
"ip授权运营", "ip衍生品", "ip授权业务", "潮玩产品", "动漫衍生品", "游戏海外发行", "游戏出海",
|
||||
"内容出海", "影视海外发行", "海外内容发行",
|
||||
"游戏研发", "数字阅读产品", "数字出版运营服务", "网络文学",
|
||||
"仓储服务", "冷链物流", "物流装备", "集装箱运输", "干散货运输", "原油运输", "港口运营", "码头运营",
|
||||
"航空客运", "旅客运输", "航空货运", "航空物流", "机场运营", "机场管理", "铁路运输", "铁路货运",
|
||||
"高速公路运营", "道路客运", "公交运营", "快递服务", "快递业务", "供应链管理服务", "综合物流服务",
|
||||
"国际货运代理", "跨境物流", "国际物流服务", "同城配送", "城市配送",
|
||||
"火力发电", "燃煤发电", "燃气发电", "水力发电", "核电运营", "新能源发电", "热电联产", "集中供热",
|
||||
"蒸汽供应", "供热业务", "配电网运营", "电网运营", "城市燃气", "城镇燃气", "电力销售", "售电业务",
|
||||
"综合能源服务", "天然气销售", "燃气销售", "燃气接驳",
|
||||
"水处理设备", "污水处理设备", "水处理膜", "除尘设备", "环卫装备", "垃圾处理设备", "固废处理设备",
|
||||
"环境监测仪器", "环境监测设备", "环境检测服务", "污水处理工程", "工业废水治理", "大气污染治理",
|
||||
"工业废气治理", "垃圾焚烧发电", "危险废物处置", "危废处理", "环卫服务", "土壤修复", "生态环境修复",
|
||||
"再生资源回收", "资源循环利用", "污水处理运营", "合同能源管理", "节能服务",
|
||||
}
|
||||
|
||||
|
||||
_PROFILE_ONLY_KEYWORDS.update(
|
||||
keyword.lower()
|
||||
for rules in (
|
||||
COAL_CHEMICAL_RULES,
|
||||
INFRASTRUCTURE_RULES,
|
||||
ELECTRONIC_COMPONENT_RULES,
|
||||
COMMUNICATION_NETWORK_RULES,
|
||||
AUTO_AFTERMARKET_RULES,
|
||||
REAL_ESTATE_RULES,
|
||||
BUILDING_MATERIAL_RULES,
|
||||
STEEL_RULES,
|
||||
NONFERROUS_RULES,
|
||||
FINANCE_RULES,
|
||||
BUSINESS_INDUSTRY_RULES,
|
||||
)
|
||||
for _, keywords in rules
|
||||
for keyword in keywords
|
||||
)
|
||||
|
||||
|
||||
def _business_keyword_count(text_value: str, keyword: str) -> int:
|
||||
text_lower = text_value.lower()
|
||||
keyword_lower = keyword.lower()
|
||||
count = 0
|
||||
for match in re.finditer(re.escape(keyword_lower), text_lower):
|
||||
clause_start = max(text_lower.rfind(mark, 0, match.start()) for mark in "。！？；，,\n") + 1
|
||||
clause_ends = [text_lower.find(mark, match.end()) for mark in "。！？；，,\n"]
|
||||
clause_end = min((index for index in clause_ends if index >= 0), default=len(text_lower))
|
||||
clause = text_lower[clause_start:clause_end]
|
||||
before = text_lower[clause_start:match.start()]
|
||||
after = text_lower[match.end():clause_end]
|
||||
if any(marker in clause for marker in _WEAK_BUSINESS_CLAUSE_MARKERS):
|
||||
continue
|
||||
if any(marker in before for marker in _DOWNSTREAM_CONTEXT_MARKERS):
|
||||
continue
|
||||
if any(after.lstrip().startswith(marker) for marker in _EQUIPMENT_SUFFIX_MARKERS):
|
||||
continue
|
||||
count += 1
|
||||
return count
|
||||
|
||||
|
||||
def classify_chain_profiles(
|
||||
frame,
|
||||
spec: ChainSpec,
|
||||
evidence_text: str = "",
|
||||
*,
|
||||
allow_default: bool = True,
|
||||
) -> list[str]:
|
||||
"""保留所有有证据的归属，优先主营证据，并移除已命中子节点的父节点。"""
|
||||
texts = _profile_texts(frame)
|
||||
concept_text, f10_profile_texts = _split_f10_profile_evidence(evidence_text)
|
||||
concept_text = concept_text.lower()
|
||||
f10_profile_texts = tuple(text_value.lower() for text_value in f10_profile_texts)
|
||||
weighted_texts = tuple(zip(
|
||||
(*texts, concept_text, *f10_profile_texts),
|
||||
(6, 3, 3, 0, 2, 8, 4, 4, 0),
|
||||
(True, True, True, True, False, True, True, True, True),
|
||||
))
|
||||
|
||||
def score(keywords: tuple[str, ...], *, profile_only: bool = False) -> int:
|
||||
return sum(
|
||||
_business_keyword_count(text_value, keyword) * weight * min(max(len(keyword) // 4, 1), 4)
|
||||
for text_value, weight, is_profile in weighted_texts
|
||||
for keyword in keywords
|
||||
if not profile_only or is_profile
|
||||
if is_profile or keyword.lower() not in _PROFILE_ONLY_KEYWORDS
|
||||
)
|
||||
|
||||
profile_scores = [score(keywords, profile_only=True) for _, keywords in spec.rules]
|
||||
use_profile_scores = any(value > 0 for value in profile_scores)
|
||||
scored = [
|
||||
(profile_scores[index] if use_profile_scores else score(keywords), -index, code)
|
||||
for index, (code, keywords) in enumerate(spec.rules)
|
||||
]
|
||||
positive = [item for item in scored if item[0] > 0]
|
||||
parent_by_code = {code: parent_code for code, _, parent_code, _ in spec.definitions}
|
||||
|
||||
def descends_from(code: str, ancestor: str) -> bool:
|
||||
seen: set[str] = set()
|
||||
parent = parent_by_code.get(code, "")
|
||||
while parent and parent not in seen:
|
||||
if parent == ancestor:
|
||||
return True
|
||||
seen.add(parent)
|
||||
parent = parent_by_code.get(parent, "")
|
||||
return False
|
||||
|
||||
specific = [
|
||||
item for item in positive
|
||||
if not any(descends_from(other_code, item[2]) for _, _, other_code in positive)
|
||||
]
|
||||
# OTHER/GENERAL can name real sibling businesses (wheat seeds, general hospitals,
|
||||
# antimony products). Never discard them just because another product matched.
|
||||
# Chip-design's generic keyword rule is only a fallback within that same branch.
|
||||
if any(code != "CHAIN_SEMI_MID_DESIGN_OTHER" and descends_from(code, "CHAIN_SEMI_MID_DESIGN") for _, _, code in specific):
|
||||
specific = [item for item in specific if item[2] != "CHAIN_SEMI_MID_DESIGN_OTHER"]
|
||||
if specific:
|
||||
return list(dict.fromkeys(code for _, _, code in sorted(specific, reverse=True)))
|
||||
return [spec.default_code] if allow_default and spec.default_code else []
|
||||
|
||||
|
||||
def classify_chain_profile(
|
||||
frame,
|
||||
spec: ChainSpec,
|
||||
evidence_text: str = "",
|
||||
*,
|
||||
allow_default: bool = True,
|
||||
) -> str | None:
|
||||
"""兼容需要单个最佳归属的调用方；同步流程使用完整归属列表。"""
|
||||
codes = classify_chain_profiles(frame, spec, evidence_text, allow_default=allow_default)
|
||||
preferred = [code for code in codes if not code.endswith(("_GENERAL", "_COMPREHENSIVE", "_OTHER"))]
|
||||
if preferred:
|
||||
return preferred[0]
|
||||
return codes[0] if codes else None
|
||||
|
||||
|
||||
def classify_semiconductor_profile(frame) -> str:
|
||||
return classify_chain_profile(frame, CHAIN_SPECS[0]) or "CHAIN_SEMI_MID_OTHER"
|
||||
@@ -0,0 +1,110 @@
|
||||
"""Pure F10 HTML parsing and disclosed-business text normalization."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from io import StringIO
|
||||
import re
|
||||
|
||||
import pandas as pd
|
||||
from bs4 import BeautifulSoup
|
||||
|
||||
F10_PROFILE_MARKER = "【F10主营介绍】"
|
||||
|
||||
|
||||
def _normalize_f10_evidence_text(value: str) -> str:
|
||||
sentences = re.split(r"(?<=[。！？；])\s*", re.sub(r"\s+", " ", str(value or "")).strip())
|
||||
non_business_fragments = ("重大资产重组", "并购重组", "借壳上市", "股份转让协议", "股权转让协议", "表决权委托协议")
|
||||
normalized = (
|
||||
sentence.strip()
|
||||
for sentence in sentences
|
||||
if sentence.strip() and not any(fragment in sentence for fragment in non_business_fragments)
|
||||
)
|
||||
return " ".join(dict.fromkeys(normalized))
|
||||
|
||||
|
||||
def parse_ths_concept_evidence_html(html: str) -> str:
|
||||
return _normalize_f10_evidence_text(" ".join(
|
||||
item["explanation"]
|
||||
for item in parse_ths_concepts_html(html)
|
||||
if item["explanation"]
|
||||
))
|
||||
|
||||
|
||||
def _is_valid_f10_concept_name(value: str) -> bool:
|
||||
name = str(value or "").strip()
|
||||
return bool(name) and len(name) <= 40 and not any(mark in name for mark in ("。", "；", ";", "！", "!", "？", "?", "\n"))
|
||||
|
||||
|
||||
def parse_ths_concepts_html(html: str) -> list[dict[str, str]]:
|
||||
concepts: list[dict[str, str]] = []
|
||||
seen: set[str] = set()
|
||||
for table in pd.read_html(StringIO(html)):
|
||||
if "概念解析" not in table.columns:
|
||||
continue
|
||||
for _, row in table.iterrows():
|
||||
text_value = str(row.get("概念解析") or "").replace("展开", " ").strip()
|
||||
if text_value and text_value.lower() != "nan":
|
||||
concept_name = str(row.get("概念名称") or "").strip()
|
||||
if _is_valid_f10_concept_name(concept_name) and concept_name.lower() != "nan" and concept_name not in seen:
|
||||
concepts.append({"name": concept_name, "explanation": text_value})
|
||||
seen.add(concept_name)
|
||||
return concepts
|
||||
|
||||
|
||||
def _decode_ths_f10_html(content: bytes) -> str:
|
||||
"""同花顺 F10 页面会在 UTF-8 与 GBK 之间切换，按业务字段完整度选择解码。"""
|
||||
markers = ("主营业务", "产品类型", "产品名称", "经营范围", "概念解析", "概念名称")
|
||||
candidates = []
|
||||
for encoding in ("utf-8", "gb18030"):
|
||||
text_value = content.decode(encoding, errors="replace")
|
||||
marker_score = sum(text_value.count(marker) for marker in markers) * 1000
|
||||
replacement_penalty = text_value.count("�") * 100
|
||||
candidates.append((marker_score - replacement_penalty, text_value))
|
||||
return max(candidates, key=lambda item: item[0])[1]
|
||||
|
||||
|
||||
def parse_ths_operate_profile_html(html: str) -> dict[str, str]:
|
||||
soup = BeautifulSoup(html, "html.parser")
|
||||
intro = soup.select_one("#intro")
|
||||
if intro is None:
|
||||
return {}
|
||||
text_value = re.sub(r"\s+", " ", intro.get_text(" ", strip=True))
|
||||
fields = ("主营业务", "产品类型", "产品名称", "经营范围")
|
||||
boundary = "|".join(fields)
|
||||
patterns = {
|
||||
field: rf"{field}\s*[：:]\s*(.*?)(?=\s*(?:{boundary})\s*[：:]|$)"
|
||||
for field in fields
|
||||
}
|
||||
return {
|
||||
field: match.group(1).strip(" 、；")
|
||||
for field, pattern in patterns.items()
|
||||
if (match := re.search(pattern, text_value, flags=re.S)) and match.group(1).strip(" 、；")
|
||||
}
|
||||
|
||||
|
||||
def _format_ths_operate_profile(profile: dict[str, str]) -> str:
|
||||
if not profile:
|
||||
return F10_PROFILE_MARKER
|
||||
fields = "；".join(f"{field}：{profile[field]}" for field in ("主营业务", "产品类型", "产品名称", "经营范围") if profile.get(field))
|
||||
return f"{F10_PROFILE_MARKER}{fields}"
|
||||
|
||||
|
||||
def _profile_texts(frame) -> tuple[str, str, str, str]:
|
||||
if frame is None or frame.empty:
|
||||
return "", "", "", ""
|
||||
row = frame.to_dict("records")[0]
|
||||
return tuple(str(row.get(column) or "").strip().lower() for column in ("主营业务", "产品类型", "产品名称", "经营范围"))
|
||||
|
||||
|
||||
def _split_f10_profile_evidence(evidence_text: str) -> tuple[str, tuple[str, str, str, str]]:
|
||||
if F10_PROFILE_MARKER not in evidence_text:
|
||||
return evidence_text, ("", "", "", "")
|
||||
concept_text, profile_text = evidence_text.split(F10_PROFILE_MARKER, 1)
|
||||
fields = []
|
||||
field_names = ("主营业务", "产品类型", "产品名称", "经营范围")
|
||||
field_boundary = "|".join(field_names)
|
||||
for field_name in field_names:
|
||||
pattern = rf"{field_name}：\s*(.*?)(?=\s*；\s*(?:{field_boundary})：|$)"
|
||||
match = re.search(pattern, profile_text, flags=re.S)
|
||||
fields.append(match.group(1).strip(" ；") if match else "")
|
||||
return concept_text, tuple(fields)
|
||||
@@ -0,0 +1,17 @@
|
||||
"""Immutable description of one industry taxonomy."""
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
INDUSTRY_CHAIN_SOURCE = "industry_chain"
|
||||
|
||||
|
||||
@dataclass(frozen=True)
|
||||
class ChainSpec:
|
||||
key: str
|
||||
source: str
|
||||
board_kind: str
|
||||
board_code: str
|
||||
definitions: tuple[tuple[str, str, str, int], ...]
|
||||
rules: tuple[tuple[str, tuple[str, ...]], ...]
|
||||
default_code: str | None = None
|
||||
additional_boards: tuple[tuple[str, str], ...] = ()
|
||||
+23
-83
@@ -1,18 +1,14 @@
|
||||
"""A股数据抓取工具主入口
|
||||
"""A股数据抓取工具兼容主入口。
|
||||
|
||||
用法示例：
|
||||
python -m src.main --stock-info # 先抓取股票列表（AKShare）
|
||||
python -m src.main --trading-day # 抓取交易日历（AKShare）
|
||||
python -m src.main --daily --start-date 20260501 --end-date 20260508 # BaoStock 日线接口
|
||||
python -m src.main --sector # 概念板块及成分股
|
||||
python -m src.main --market-daily # 汇总每日涨跌停统计（依赖 stock_daily）
|
||||
python -m src.main --sina-min1 # 新浪 1 分钟数据，默认全市场，默认 2010 至今
|
||||
python -m src.main --sina-min1 --symbol 600519 --start-date 20180101 --end-date 20260517
|
||||
保留 `src.main` 作为旧 CLI/测试 patch 点：
|
||||
- `build_parser` / `selected_commands` / `run_command` 复用新的 app 层 helper
|
||||
- `load_config` / `init_db` 继续暴露在当前模块，方便旧测试 patch
|
||||
- `main()` 在当前模块内完成启动流程，确保 patch.object(src.main, ...) 生效
|
||||
"""
|
||||
|
||||
import argparse
|
||||
|
||||
from src.config import load_config
|
||||
from src.app.bootstrap import execute_cli
|
||||
from src.app.commands import build_parser, run_command, selected_commands
|
||||
from src.core.config import load_config
|
||||
from src.db import init_db
|
||||
from src.log import get_logger
|
||||
|
||||
@@ -20,80 +16,24 @@ _logger = get_logger("main")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(description="A股数据抓取工具")
|
||||
parser.add_argument("--stock-info", action="store_true", help="抓取股票列表（AKShare）")
|
||||
parser.add_argument("--trading-day", action="store_true", help="抓取交易日历（AKShare）")
|
||||
daily_group = parser.add_mutually_exclusive_group()
|
||||
daily_group.add_argument("--daily", action="store_true", help="抓取日线行情（BaoStock 日线接口；只写入 stock_daily，不更新停牌/无数据记录）")
|
||||
daily_group.add_argument(
|
||||
"--daily-no-data-only",
|
||||
action="store_true",
|
||||
help="仅更新停牌/无数据记录，不写入 stock_daily（BaoStock 日线接口）",
|
||||
execute_cli(
|
||||
build_parser=build_parser,
|
||||
selected_commands=selected_commands,
|
||||
load_config=load_config,
|
||||
init_db=init_db,
|
||||
run_command=run_command,
|
||||
logger=_logger,
|
||||
)
|
||||
parser.add_argument("--sector", action="store_true", help="抓取概念板块及成分股")
|
||||
parser.add_argument("--sina-min1", action="store_true",
|
||||
help="抓取新浪 1 分钟数据并写入 stock_min1（默认全市场，默认 2010 至今，可配合 --symbol / --start-date / --end-date）")
|
||||
parser.add_argument("--market-daily", action="store_true",
|
||||
help="汇总每日涨跌停统计（从 stock_daily 聚合）")
|
||||
parser.add_argument("--start-date", type=str, help="开始日期 YYYYMMDD")
|
||||
parser.add_argument("--end-date", type=str, help="结束日期 YYYYMMDD")
|
||||
parser.add_argument("--symbol", type=str, help="指定单只或多个股票代码，逗号分隔；不传则抓取全市场，供 --sina-min1 使用")
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if not any([args.stock_info, args.trading_day, args.daily,
|
||||
args.daily_no_data_only, args.sector, args.sina_min1, args.market_daily]):
|
||||
parser.print_help()
|
||||
return
|
||||
|
||||
load_config()
|
||||
|
||||
init_db()
|
||||
|
||||
# market_daily 只读 stock_daily，提前处理
|
||||
if args.market_daily:
|
||||
from src.fetchers.market_daily import fetch_market_daily
|
||||
fetch_market_daily(start_date=args.start_date, end_date=args.end_date)
|
||||
# 若仅运行 market-daily，避免无谓的登录
|
||||
if not any([args.stock_info, args.trading_day, args.daily,
|
||||
args.daily_no_data_only, args.sector, args.sina_min1]):
|
||||
_logger.info("全部任务完成")
|
||||
return
|
||||
|
||||
# sector 不需要额外网络数据源，提前处理
|
||||
if args.sector:
|
||||
from src.fetchers.sector import fetch_sector
|
||||
fetch_sector()
|
||||
if not any([args.stock_info, args.trading_day, args.daily,
|
||||
args.daily_no_data_only, args.market_daily]):
|
||||
_logger.info("全部任务完成")
|
||||
return
|
||||
|
||||
if args.stock_info:
|
||||
from src.fetchers.stock_list import fetch_stock_list
|
||||
fetch_stock_list()
|
||||
|
||||
if args.trading_day:
|
||||
from src.fetchers.trading_day import fetch_trading_days
|
||||
fetch_trading_days(start_date=args.start_date, end_date=args.end_date)
|
||||
|
||||
if args.daily or args.daily_no_data_only:
|
||||
from src.fetchers.daily import fetch_daily
|
||||
fetch_daily(
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
no_data_only=args.daily_no_data_only,
|
||||
)
|
||||
|
||||
if args.sina_min1:
|
||||
from src.fetchers.sina_minute import fetch_sina_min1
|
||||
fetch_sina_min1(
|
||||
symbol=args.symbol,
|
||||
start_date=args.start_date,
|
||||
end_date=args.end_date,
|
||||
)
|
||||
|
||||
_logger.info("全部任务完成")
|
||||
__all__ = [
|
||||
"build_parser",
|
||||
"selected_commands",
|
||||
"run_command",
|
||||
"load_config",
|
||||
"init_db",
|
||||
"main",
|
||||
]
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
|
||||
@@ -0,0 +1,21 @@
|
||||
"""业务流程层。"""
|
||||
|
||||
from src.services.backfill_service import backfill_once, run_backfill
|
||||
from src.services.industry_service import sync_industry_tree
|
||||
from src.services.index_service import sync_index_daily
|
||||
from src.services.daily_service import sync_daily
|
||||
from src.services.minute_service import sync_minute
|
||||
from src.services.reference_service import sync_stock_info, sync_trading_days
|
||||
from src.services.trade_status_service import archive_trade_status
|
||||
|
||||
__all__ = [
|
||||
"archive_trade_status",
|
||||
"backfill_once",
|
||||
"run_backfill",
|
||||
"sync_industry_tree",
|
||||
"sync_index_daily",
|
||||
"sync_daily",
|
||||
"sync_minute",
|
||||
"sync_stock_info",
|
||||
"sync_trading_days",
|
||||
]
|
||||
@@ -0,0 +1,320 @@
|
||||
"""股票日 K、分钟 K 与指数日线历史缺口的自动补全流程。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import time
|
||||
from datetime import date, datetime, timedelta
|
||||
|
||||
from sqlalchemy import func, select
|
||||
|
||||
from src.config import get_fetch_config
|
||||
from src.db import StockDaily, get_session, get_stock_minute_model
|
||||
from src.log import get_logger
|
||||
from src.services.daily_service import sync_daily
|
||||
from src.services.index_service import sync_index_daily
|
||||
from src.services.minute_service import sync_minute
|
||||
|
||||
_logger = get_logger("backfill")
|
||||
|
||||
DEFAULT_BACKFILL_START_DATE = "20150101"
|
||||
DEFAULT_BACKFILL_INTERVAL_SECONDS = 3600
|
||||
DEFAULT_BACKFILL_LOOKBACK_DAYS = 30
|
||||
BACKFILL_MODES = {"auto", "incremental", "full"}
|
||||
|
||||
|
||||
def _today() -> str:
|
||||
return datetime.now().strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _configured_start_date() -> str:
|
||||
try:
|
||||
value = get_fetch_config().get("backfill_start_date", DEFAULT_BACKFILL_START_DATE)
|
||||
except Exception:
|
||||
value = DEFAULT_BACKFILL_START_DATE
|
||||
return str(value or DEFAULT_BACKFILL_START_DATE)
|
||||
|
||||
|
||||
def _configured_interval() -> int:
|
||||
try:
|
||||
value = int(get_fetch_config().get("backfill_interval_seconds", DEFAULT_BACKFILL_INTERVAL_SECONDS))
|
||||
except (TypeError, ValueError):
|
||||
value = DEFAULT_BACKFILL_INTERVAL_SECONDS
|
||||
return max(value, 1)
|
||||
|
||||
|
||||
def _configured_lookback_days() -> int:
|
||||
try:
|
||||
value = int(get_fetch_config().get("backfill_incremental_lookback_days", DEFAULT_BACKFILL_LOOKBACK_DAYS))
|
||||
except (TypeError, ValueError):
|
||||
value = DEFAULT_BACKFILL_LOOKBACK_DAYS
|
||||
return max(value, 1)
|
||||
|
||||
|
||||
def _validate_date(value: str, name: str) -> str:
|
||||
try:
|
||||
datetime.strptime(value, "%Y%m%d")
|
||||
except (TypeError, ValueError):
|
||||
raise ValueError(f"{name} 必须是 YYYYMMDD 格式: {value}") from None
|
||||
return value
|
||||
|
||||
|
||||
def _resolve_date_range(start_date: str | None, end_date: str | None) -> tuple[str, str]:
|
||||
start = _validate_date(start_date or _configured_start_date(), "开始日期")
|
||||
end = _validate_date(end_date or _today(), "结束日期")
|
||||
if start > end:
|
||||
raise ValueError(f"开始日期不能晚于结束日期: {start} > {end}")
|
||||
return start, end
|
||||
|
||||
|
||||
def _normalize_symbols(symbol: str | None) -> list[str]:
|
||||
if not symbol:
|
||||
return []
|
||||
|
||||
symbols: list[str] = []
|
||||
for item in symbol.replace("，", ",").split(","):
|
||||
value = item.strip()
|
||||
lower = value.lower()
|
||||
if lower.startswith(("sh.", "sz.")):
|
||||
value = value[3:]
|
||||
elif lower.startswith(("sh", "sz")):
|
||||
value = value[2:].lstrip(".")
|
||||
elif "." in value:
|
||||
base, suffix = value.upper().split(".", 1)
|
||||
if suffix in {"SH", "SZ"}:
|
||||
value = base
|
||||
if value and value not in symbols:
|
||||
symbols.append(value)
|
||||
return symbols
|
||||
|
||||
|
||||
def _as_date(value) -> date | None:
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, datetime):
|
||||
return value.date()
|
||||
if isinstance(value, date):
|
||||
return value
|
||||
try:
|
||||
return datetime.strptime(str(value)[:10], "%Y-%m-%d").date()
|
||||
except ValueError:
|
||||
return None
|
||||
|
||||
|
||||
def _latest_daily_date(symbols: list[str], end_date: str) -> date | None:
|
||||
end_day = datetime.strptime(end_date, "%Y%m%d").date()
|
||||
session = get_session()
|
||||
try:
|
||||
statement = select(func.max(StockDaily.date)).where(StockDaily.date <= end_day)
|
||||
if symbols:
|
||||
statement = statement.where(StockDaily.code.in_(symbols))
|
||||
return _as_date(session.execute(statement).scalar())
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _latest_minute_date(freq: int, symbols: list[str], end_date: str) -> date | None:
|
||||
model_cls = get_stock_minute_model(freq)
|
||||
end_day = datetime.strptime(end_date, "%Y%m%d").date()
|
||||
next_day = datetime.combine(end_day + timedelta(days=1), datetime.min.time())
|
||||
session = get_session()
|
||||
try:
|
||||
statement = select(func.max(model_cls.datetime)).where(model_cls.datetime < next_day)
|
||||
if symbols:
|
||||
statement = statement.where(model_cls.code.in_(symbols))
|
||||
return _as_date(session.execute(statement).scalar())
|
||||
finally:
|
||||
session.close()
|
||||
|
||||
|
||||
def _incremental_start_date(
|
||||
latest_date: date | None,
|
||||
*,
|
||||
end_date: str,
|
||||
configured_start: str,
|
||||
lookback_days: int,
|
||||
) -> str | None:
|
||||
if latest_date is None:
|
||||
return None
|
||||
|
||||
end_day = datetime.strptime(end_date, "%Y%m%d").date()
|
||||
latest_date = min(latest_date, end_day)
|
||||
candidate = latest_date - timedelta(days=max(int(lookback_days), 1))
|
||||
lower_bound = datetime.strptime(configured_start, "%Y%m%d").date()
|
||||
return max(candidate, lower_bound).strftime("%Y%m%d")
|
||||
|
||||
|
||||
def _resolve_dataset_start(
|
||||
*,
|
||||
dataset: str,
|
||||
mode: str,
|
||||
explicit_start: str | None,
|
||||
configured_start: str,
|
||||
end_date: str,
|
||||
symbols: list[str],
|
||||
lookback_days: int,
|
||||
) -> str:
|
||||
if explicit_start:
|
||||
return explicit_start
|
||||
if mode == "full":
|
||||
return configured_start
|
||||
|
||||
if dataset == "daily":
|
||||
latest = _latest_daily_date(symbols, end_date)
|
||||
else:
|
||||
latest = _latest_minute_date(5, symbols, end_date)
|
||||
|
||||
incremental_start = _incremental_start_date(
|
||||
latest,
|
||||
end_date=end_date,
|
||||
configured_start=configured_start,
|
||||
lookback_days=lookback_days,
|
||||
)
|
||||
if incremental_start is None:
|
||||
_logger.info("%s 未找到已有数据，当前轮回退为历史全量补全", dataset)
|
||||
return configured_start
|
||||
return incremental_start
|
||||
|
||||
|
||||
def backfill_once(
|
||||
*,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
mode: str = "auto",
|
||||
lookback_days: int | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
"""执行一轮历史缺口补全，包括股票日 K、分钟 K 和常用指数日线。"""
|
||||
if mode not in BACKFILL_MODES:
|
||||
raise ValueError(f"不支持的补全模式: {mode}，可选 auto/incremental/full")
|
||||
|
||||
if start_date:
|
||||
_, end = _resolve_date_range(start_date, end_date)
|
||||
else:
|
||||
_, end = _resolve_date_range(None, end_date)
|
||||
configured_start = _validate_date(_configured_start_date(), "历史起始日期")
|
||||
explicit_start = _validate_date(start_date, "开始日期") if start_date else None
|
||||
symbols = _normalize_symbols(symbol)
|
||||
effective_lookback = max(int(lookback_days or _configured_lookback_days()), 1)
|
||||
daily_start = _resolve_dataset_start(
|
||||
dataset="daily",
|
||||
mode=mode,
|
||||
explicit_start=explicit_start,
|
||||
configured_start=configured_start,
|
||||
end_date=end,
|
||||
symbols=symbols,
|
||||
lookback_days=effective_lookback,
|
||||
)
|
||||
minute_start = _resolve_dataset_start(
|
||||
dataset="minute",
|
||||
mode=mode,
|
||||
explicit_start=explicit_start,
|
||||
configured_start=configured_start,
|
||||
end_date=end,
|
||||
symbols=symbols,
|
||||
lookback_days=effective_lookback,
|
||||
)
|
||||
_logger.info(
|
||||
"开始自动补全历史数据，日K区间:%s ~ %s，5分钟K区间:%s ~ %s，模式:%s",
|
||||
daily_start,
|
||||
end,
|
||||
minute_start,
|
||||
end,
|
||||
mode,
|
||||
)
|
||||
|
||||
errors: list[tuple[str, Exception]] = []
|
||||
try:
|
||||
sync_daily(
|
||||
start_date=daily_start,
|
||||
end_date=end,
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
skip_suspended=skip_suspended,
|
||||
)
|
||||
except KeyboardInterrupt:
|
||||
raise
|
||||
except Exception as exc:
|
||||
errors.append(("日K", exc))
|
||||
_logger.exception("日K历史缺口补全失败，将继续补分钟K: %s", exc)
|
||||
|
||||
try:
|
||||
sync_minute(
|
||||
symbol=symbol,
|
||||
start_date=minute_start,
|
||||
end_date=end,
|
||||
limit=limit,
|
||||
skip_suspended=skip_suspended,
|
||||
)
|
||||
except KeyboardInterrupt:
|
||||
raise
|
||||
except Exception as exc:
|
||||
errors.append(("分钟K", exc))
|
||||
_logger.exception("分钟K历史缺口补全失败，将继续补指数日线: %s", exc)
|
||||
|
||||
try:
|
||||
sync_index_daily(start_date=daily_start, end_date=end)
|
||||
except KeyboardInterrupt:
|
||||
raise
|
||||
except Exception as exc:
|
||||
errors.append(("指数日线", exc))
|
||||
_logger.exception("指数日线历史缺口补全失败: %s", exc)
|
||||
|
||||
if errors:
|
||||
failed = ", ".join(name for name, _ in errors)
|
||||
raise RuntimeError(f"历史数据补全部分失败: {failed}") from errors[0][1]
|
||||
|
||||
_logger.info("本轮历史数据补全完成")
|
||||
|
||||
|
||||
def run_backfill(
|
||||
*,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
daemon: bool = False,
|
||||
interval_seconds: int | None = None,
|
||||
mode: str = "auto",
|
||||
lookback_days: int | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
"""运行一次补全，或以固定间隔持续扫描历史缺口。"""
|
||||
interval = max(int(interval_seconds or _configured_interval()), 1)
|
||||
cycle = 0
|
||||
while True:
|
||||
cycle += 1
|
||||
_logger.info("历史数据自动补全第 %d 轮%s", cycle, "（守护模式）" if daemon else "")
|
||||
try:
|
||||
backfill_once(
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
mode=mode,
|
||||
lookback_days=lookback_days,
|
||||
skip_suspended=skip_suspended,
|
||||
)
|
||||
except KeyboardInterrupt:
|
||||
raise
|
||||
except Exception:
|
||||
if not daemon:
|
||||
raise
|
||||
_logger.exception("第 %d 轮历史数据补全失败，下一轮继续重试", cycle)
|
||||
|
||||
if not daemon:
|
||||
return
|
||||
|
||||
_logger.info("本轮结束，%d 秒后重新扫描股票日K、分钟K和指数日线缺口", interval)
|
||||
time.sleep(interval)
|
||||
|
||||
|
||||
__all__ = [
|
||||
"DEFAULT_BACKFILL_START_DATE",
|
||||
"DEFAULT_BACKFILL_INTERVAL_SECONDS",
|
||||
"DEFAULT_BACKFILL_LOOKBACK_DAYS",
|
||||
"BACKFILL_MODES",
|
||||
"backfill_once",
|
||||
"run_backfill",
|
||||
]
|
||||
@@ -0,0 +1,25 @@
|
||||
"""日线抓取业务流程。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.fetchers.tdx_daily import fetch_tdx_daily
|
||||
|
||||
|
||||
def sync_daily(
|
||||
*,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
fetch_tdx_daily(
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
skip_suspended=skip_suspended,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["sync_daily"]
|
||||
@@ -0,0 +1,10 @@
|
||||
"""指数行情同步流程。"""
|
||||
|
||||
from src.fetchers.index_daily import fetch_index_daily
|
||||
|
||||
|
||||
def sync_index_daily(start_date: str | None = None, end_date: str | None = None) -> int:
|
||||
return fetch_index_daily(start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
__all__ = ["sync_index_daily"]
|
||||
@@ -0,0 +1,26 @@
|
||||
"""产业链同步服务。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.fetchers.industry_chain import sync_industry_chains
|
||||
|
||||
|
||||
def sync_industry_tree(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
limit: int | None = None,
|
||||
taxonomy_only: bool = False,
|
||||
refresh: bool = False,
|
||||
delay_seconds: float | None = None,
|
||||
) -> tuple[int, int, int]:
|
||||
"""同步自动产业链和股票归属，返回目录数、成功归属数、失败数。"""
|
||||
_ = delay_seconds
|
||||
return sync_industry_chains(
|
||||
symbol=symbol,
|
||||
limit=limit,
|
||||
taxonomy_only=taxonomy_only,
|
||||
refresh=refresh,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["sync_industry_tree"]
|
||||
@@ -0,0 +1,25 @@
|
||||
"""分钟 K 抓取业务流程。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.fetchers.tdx_minute import fetch_tdx_minute
|
||||
|
||||
|
||||
def sync_minute(
|
||||
*,
|
||||
symbol: str | None = None,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
limit: int | None = None,
|
||||
skip_suspended: bool = True,
|
||||
) -> None:
|
||||
fetch_tdx_minute(
|
||||
symbol=symbol,
|
||||
start_date=start_date,
|
||||
end_date=end_date,
|
||||
limit=limit,
|
||||
skip_suspended=skip_suspended,
|
||||
)
|
||||
|
||||
|
||||
__all__ = ["sync_minute"]
|
||||
@@ -0,0 +1,16 @@
|
||||
"""基础参考数据同步业务流程。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.fetchers.tdx_daily import bootstrap_stock_list, fetch_trading_days
|
||||
|
||||
|
||||
def sync_stock_info() -> list[str]:
|
||||
return bootstrap_stock_list()
|
||||
|
||||
|
||||
def sync_trading_days(start_date: str, end_date: str) -> list[str]:
|
||||
return fetch_trading_days(start_date, end_date)
|
||||
|
||||
|
||||
__all__ = ["sync_stock_info", "sync_trading_days"]
|
||||
@@ -0,0 +1,16 @@
|
||||
"""交易状态归档业务流程。"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from src.fetchers.trade_status import fetch_trade_status
|
||||
|
||||
|
||||
def archive_trade_status(
|
||||
*,
|
||||
start_date: str | None = None,
|
||||
end_date: str | None = None,
|
||||
) -> None:
|
||||
fetch_trade_status(start_date=start_date, end_date=end_date)
|
||||
|
||||
|
||||
__all__ = ["archive_trade_status"]
|
||||
@@ -0,0 +1 @@
|
||||
"""数据源适配层。"""
|
||||
@@ -0,0 +1,5 @@
|
||||
"""兼容导出：后续迁移股票列表数据源。"""
|
||||
|
||||
from src.fetchers.stock_list import fetch_stock_list
|
||||
|
||||
__all__ = ["fetch_stock_list"]
|
||||
@@ -0,0 +1,5 @@
|
||||
"""兼容导出：后续迁移交易日历数据源。"""
|
||||
|
||||
from src.fetchers.trading_day import fetch_trading_days, get_trading_days
|
||||
|
||||
__all__ = ["fetch_trading_days", "get_trading_days"]
|
||||
+26
-18
@@ -1,27 +1,35 @@
|
||||
# tests 目录
|
||||
|
||||
针对纯函数、BaoStock 日线、AKShare 交易日/股票列表和 TDX 本地逻辑的单元测试。
|
||||
这里的测试覆盖 CLI、行情抓取、兼容入口和数据库安全护栏。
|
||||
|
||||
## 运行
|
||||
|
||||
```bash
|
||||
pip install -e .[dev] # 或 pip install pytest
|
||||
pytest -v
|
||||
pip install -e ".[dev]" # 或至少安装 pytest
|
||||
pytest -q
|
||||
```
|
||||
|
||||
## 覆盖范围（66 个用例）
|
||||
## 当前覆盖主题
|
||||
|
||||
- `test_daily_derived.py` —— `daily._fill_derived_fields` 振幅/涨跌幅补算（5）
|
||||
- `test_daily_source_codes.py` —— `daily._ak_hist_to_rows` 日线转换（1）
|
||||
- `test_daily_sources_http.py` —— `daily._fetch_one_stock` / `daily.fetch_daily` BaoStock 日线抓取（3）
|
||||
- `test_stock_list.py` —— `stock_list.fetch_stock_list` / AKShare 股票列表读取（2）
|
||||
- `test_intraday_tdx.py` —— `intraday.fetch_intraday` / 通达信分钟转换（6）
|
||||
- `test_log.py` —— `log.get_logger` 命名空间、handler 幂等、env 控制 level（5）
|
||||
- `test_market_classification.py` —— `market_daily._is_20pct` 板块判定（4）
|
||||
- `test_main_sina_minute.py` —— 新浪 1 分钟命令入口（3）
|
||||
- `test_trading_day.py` —— `trading_day.get_trading_days` / AKShare 交易日历读取（2）
|
||||
- `test_sector_http.py` —— 通达信概念板块名称/成分股解析（2）
|
||||
- `test_sector_merge.py` —— `sector.fetch_sector` 通达信增量跳过保护（2）
|
||||
- `test_sina_minute.py` —— 新浪 1 分钟转换与落库（5）
|
||||
- `test_tdx_blocks.py` —— 通达信本地板块文件解析（2）
|
||||
- `test_tdx_client.py` —— 通达信客户端适配与初始化包解压（11）
|
||||
- 日线：转换、股票列表、交易日历、批量日线入口
|
||||
- 分钟：5 分钟 K 转换、写入及 `--minute` 命令分发
|
||||
- `services`：CLI 到业务流程层的转发边界
|
||||
- `trade_status` / `suspended_markers`：交易/停牌/未上市/数据异常归档，以及日K/分钟K跳过已标记停牌日
|
||||
- `stock_list` / `trading_day`：旧路径兼容入口
|
||||
- `db_schema` / `no_delete_policy`：`stock_daily` 非空约束、分块 upsert、禁止 `DELETE` / `DROP` / `ALTER TABLE`
|
||||
- `industry_chain`：产品节点多归属、父子去重、F10 证据解析和无效页面识别
|
||||
- `industry_memberships`：独立 SQLite 持久化验证、失效保留与恢复、批次回滚、来源锁边界
|
||||
- `config`：配置文件切换、无效 YAML、数据库凭据特殊字符编码
|
||||
|
||||
## 当前测试文件
|
||||
|
||||
- `test_db_schema.py`
|
||||
- `test_log.py`
|
||||
- `test_main_daily_mode.py`
|
||||
- `test_no_delete_policy.py`
|
||||
- `test_services.py`
|
||||
- `test_stock_list.py`
|
||||
- 日线抓取测试
|
||||
- 分钟抓取测试
|
||||
- `test_trade_status.py`
|
||||
- `test_trading_day.py`
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
from datetime import datetime
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from src.fetchers.cninfo_industry import (
|
||||
CNINFO_DETAIL_SOURCE,
|
||||
parse_cninfo_profile_industry,
|
||||
parse_cninfo_taxonomy,
|
||||
)
|
||||
|
||||
|
||||
def _aerospace_nodes():
|
||||
return [
|
||||
{"industry_code": "XC", "industry_name": "产业细分", "parent_code": "", "industry_level": 0},
|
||||
{"industry_code": "XC37", "industry_name": "高端装备", "parent_code": "XC", "industry_level": 1},
|
||||
{"industry_code": "XC3705", "industry_name": "航空航天", "parent_code": "XC37", "industry_level": 2},
|
||||
{"industry_code": "XC370502", "industry_name": "航空航天器", "parent_code": "XC3705", "industry_level": 3},
|
||||
{"industry_code": "XC37050201", "industry_name": "航天飞机", "parent_code": "XC370502", "industry_level": 4},
|
||||
{"industry_code": "XC37050202", "industry_name": "航空飞机", "parent_code": "XC370502", "industry_level": 4},
|
||||
{"industry_code": "XC37050203", "industry_name": "卫星发射装置", "parent_code": "XC370502", "industry_level": 4},
|
||||
{"industry_code": "XC37050204", "industry_name": "航空航天器零配件", "parent_code": "XC370502", "industry_level": 4},
|
||||
]
|
||||
|
||||
|
||||
def test_parse_cninfo_taxonomy_builds_single_parent_nodes_with_detail_source():
|
||||
frame = pd.DataFrame(
|
||||
[
|
||||
{"类目编码": "XC", "类目名称": "产业细分", "父类编码": "008", "分级": 0},
|
||||
{"类目编码": "XC37", "类目名称": "高端装备", "父类编码": "XC", "分级": 1},
|
||||
{"类目编码": "XC3705", "类目名称": "航空航天", "父类编码": "XC37", "分级": 2},
|
||||
]
|
||||
)
|
||||
|
||||
rows = parse_cninfo_taxonomy(frame, source=CNINFO_DETAIL_SOURCE, now=datetime(2026, 7, 12, 9, 0))
|
||||
|
||||
assert [row["industry_code"] for row in rows] == ["XC", "XC37", "XC3705"]
|
||||
assert rows[0]["parent_code"] == ""
|
||||
assert rows[-1]["parent_code"] == "XC37"
|
||||
assert rows[-1]["source"] == CNINFO_DETAIL_SOURCE
|
||||
|
||||
|
||||
def test_parse_cninfo_profile_industry_falls_back_to_disclosed_industry_without_clear_product_evidence():
|
||||
frame = pd.DataFrame(
|
||||
[{"所属行业": "航空航天", "主营业务": "高端装备研发与制造", "经营范围": "技术服务"}]
|
||||
)
|
||||
|
||||
row = parse_cninfo_profile_industry(frame, "600001", _aerospace_nodes(), now=datetime(2026, 7, 12, 9, 0))
|
||||
|
||||
assert row is not None
|
||||
assert row["source"] == CNINFO_DETAIL_SOURCE
|
||||
assert row["stock_code"] == "600001"
|
||||
assert row["industry_code"] == "XC3705"
|
||||
|
||||
|
||||
def test_parse_cninfo_profile_industry_descends_on_unique_satellite_launch_evidence():
|
||||
frame = pd.DataFrame(
|
||||
[{"所属行业": "航空航天", "主营业务": "卫星发射装置研制", "经营范围": "卫星发射装置生产"}]
|
||||
)
|
||||
|
||||
row = parse_cninfo_profile_industry(frame, "600879", _aerospace_nodes(), now=datetime(2026, 7, 12, 9, 0))
|
||||
|
||||
assert row is not None
|
||||
assert row["industry_code"] == "XC37050203"
|
||||
|
||||
|
||||
def test_parse_cninfo_profile_industry_does_not_descend_when_product_evidence_is_ambiguous():
|
||||
frame = pd.DataFrame(
|
||||
[
|
||||
{
|
||||
"所属行业": "航空航天",
|
||||
"主营业务": "卫星发射装置和航空航天器零配件制造",
|
||||
"经营范围": "卫星发射装置、航空航天器零配件生产",
|
||||
}
|
||||
]
|
||||
)
|
||||
|
||||
row = parse_cninfo_profile_industry(frame, "600002", _aerospace_nodes(), now=datetime(2026, 7, 12, 9, 0))
|
||||
|
||||
assert row is not None
|
||||
assert row["industry_code"] == "XC3705"
|
||||
@@ -0,0 +1,35 @@
|
||||
import pytest
|
||||
from sqlalchemy.engine import make_url
|
||||
|
||||
from src import config
|
||||
|
||||
|
||||
def test_mysql_credentials_with_url_characters_round_trip():
|
||||
settings = {"mysql": {"user": "reader@local", "password": "p@ss:/?#%word", "host": "localhost", "port": "3306", "database": "ashare"}}
|
||||
url = make_url(config.get_mysql_url(settings))
|
||||
assert url.username == settings["mysql"]["user"]
|
||||
assert url.password == settings["mysql"]["password"]
|
||||
assert url.host == "localhost" and url.port == 3306
|
||||
|
||||
|
||||
def test_explicit_config_path_is_not_hidden_by_previous_cache(tmp_path, monkeypatch):
|
||||
monkeypatch.setattr(config, "_CONFIG", None)
|
||||
monkeypatch.setattr(config, "_CONFIG_PATH", None)
|
||||
first, second = tmp_path / "first.yaml", tmp_path / "second.yaml"
|
||||
first.write_text("fetch:\n workers: 1\n", encoding="utf-8")
|
||||
second.write_text("fetch:\n workers: 2\n", encoding="utf-8")
|
||||
assert config.load_config(str(first))["fetch"]["workers"] == 1
|
||||
assert config.load_config(str(second))["fetch"]["workers"] == 2
|
||||
monkeypatch.setenv("ASHARE_CONFIG", str(first))
|
||||
assert config.load_config()["fetch"]["workers"] == 1
|
||||
|
||||
|
||||
@pytest.mark.parametrize("contents", ["", "- item\n", "123"])
|
||||
def test_invalid_config_is_rejected_without_replacing_cache(tmp_path, monkeypatch, contents):
|
||||
monkeypatch.setattr(config, "_CONFIG", {"fetch": {"workers": 8}})
|
||||
monkeypatch.setattr(config, "_CONFIG_PATH", tmp_path / "previous.yaml")
|
||||
path = tmp_path / "invalid.yaml"
|
||||
path.write_text(contents, encoding="utf-8")
|
||||
with pytest.raises(ValueError, match="YAML"):
|
||||
config.load_config(str(path))
|
||||
assert config._CONFIG == {"fetch": {"workers": 8}}
|
||||
@@ -1,63 +0,0 @@
|
||||
"""验证 daily._fill_derived_fields 的振幅/涨跌幅/涨跌额补算逻辑"""
|
||||
|
||||
from src.fetchers.daily import _fill_derived_fields
|
||||
|
||||
|
||||
def test_empty_input():
|
||||
assert _fill_derived_fields([]) == []
|
||||
|
||||
|
||||
def test_first_row_has_no_preclose():
|
||||
rows = [{"date": "2026-05-08", "open": 10.0, "high": 11.0, "low": 9.5, "close": 10.5,
|
||||
"amplitude": None, "pct_change": None, "change": None}]
|
||||
out = _fill_derived_fields(rows)
|
||||
# 没有 preclose 时不应补，pct_change 等保持 None
|
||||
assert out[0]["pct_change"] is None
|
||||
assert out[0]["change"] is None
|
||||
assert out[0]["amplitude"] is None
|
||||
|
||||
|
||||
def test_subsequent_row_derives_from_previous_close():
|
||||
rows = [
|
||||
{"date": "2026-05-08", "open": 10.0, "high": 11.0, "low": 9.5, "close": 10.0,
|
||||
"amplitude": None, "pct_change": None, "change": None},
|
||||
{"date": "2026-05-09", "open": 10.2, "high": 11.0, "low": 9.0, "close": 11.0,
|
||||
"amplitude": None, "pct_change": None, "change": None},
|
||||
]
|
||||
out = _fill_derived_fields(rows)
|
||||
second = out[1]
|
||||
# 涨跌额 = 11 - 10 = 1.0
|
||||
assert second["change"] == 1.0
|
||||
# 涨跌幅 = (11 - 10) / 10 * 100 = 10.0
|
||||
assert second["pct_change"] == 10.0
|
||||
# 振幅 = (11 - 9) / 10 * 100 = 20.0
|
||||
assert second["amplitude"] == 20.0
|
||||
|
||||
|
||||
def test_existing_values_are_preserved():
|
||||
"""已有值不应被覆盖"""
|
||||
rows = [
|
||||
{"date": "2026-05-08", "open": 10.0, "high": 11.0, "low": 9.5, "close": 10.0,
|
||||
"amplitude": None, "pct_change": None, "change": None},
|
||||
{"date": "2026-05-09", "open": 10.2, "high": 11.0, "low": 9.0, "close": 11.0,
|
||||
"amplitude": 99.9, "pct_change": 88.8, "change": 7.7},
|
||||
]
|
||||
out = _fill_derived_fields(rows)
|
||||
assert out[1]["amplitude"] == 99.9
|
||||
assert out[1]["pct_change"] == 88.8
|
||||
assert out[1]["change"] == 7.7
|
||||
|
||||
|
||||
def test_zero_preclose_does_not_divide_by_zero():
|
||||
"""preclose=0 时不应抛 ZeroDivisionError，保持 None"""
|
||||
rows = [
|
||||
{"date": "2026-05-08", "open": 0, "high": 0, "low": 0, "close": 0,
|
||||
"amplitude": None, "pct_change": None, "change": None},
|
||||
{"date": "2026-05-09", "open": 0.1, "high": 1, "low": 0.1, "close": 0.5,
|
||||
"amplitude": None, "pct_change": None, "change": None},
|
||||
]
|
||||
out = _fill_derived_fields(rows)
|
||||
# preclose=0 时三个派生字段都保持 None
|
||||
assert out[1]["pct_change"] is None
|
||||
assert out[1]["change"] is None
|
||||
assert out[1]["amplitude"] is None
|
||||
@@ -1,33 +0,0 @@
|
||||
"""验证 daily.py 中日线行转换逻辑。"""
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from src.fetchers.daily import _ak_hist_to_rows
|
||||
|
||||
|
||||
def test_ak_hist_to_rows_converts_dataframe():
|
||||
"""日线表应转换成可入库的日线行。"""
|
||||
hist_df = pd.DataFrame(
|
||||
{
|
||||
"日期": ["2026-05-17", "2026-05-18"],
|
||||
"开盘": [10.0, 10.2],
|
||||
"收盘": [10.3, 10.4],
|
||||
"最高": [10.5, 10.6],
|
||||
"最低": [9.9, 10.0],
|
||||
"成交量": [100, 120],
|
||||
"成交额": [1030, 1248],
|
||||
"振幅": [5.8, 5.9],
|
||||
"涨跌幅": [3.0, 0.97],
|
||||
"涨跌额": [0.3, 0.1],
|
||||
"换手率": [1.0, 1.2],
|
||||
}
|
||||
)
|
||||
|
||||
rows = _ak_hist_to_rows("600000", hist_df)
|
||||
|
||||
assert len(rows) == 2
|
||||
assert rows[0]["code"] == "600000"
|
||||
assert rows[0]["date"] == "2026-05-17"
|
||||
assert rows[0]["open"] == 10.0
|
||||
assert rows[0]["turnover"] == 1030
|
||||
assert rows[1]["date"] == "2026-05-18"
|
||||
@@ -1,304 +0,0 @@
|
||||
"""验证 daily.py 中 BaoStock 日线抓取逻辑（不打网络）。"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pandas as pd
|
||||
|
||||
from src.fetchers import daily
|
||||
|
||||
|
||||
def _make_market_data() -> dict:
|
||||
return pd.DataFrame(
|
||||
{
|
||||
"日期": ["2026-05-08", "2026-05-09"],
|
||||
"开盘": [10.0, 10.2],
|
||||
"收盘": [10.3, 10.4],
|
||||
"最高": [10.5, 10.6],
|
||||
"最低": [9.9, 10.0],
|
||||
"成交量": [100, 120],
|
||||
"成交额": [1030, 1248],
|
||||
"振幅": [5.8, 5.9],
|
||||
"涨跌幅": [3.0, 0.97],
|
||||
"涨跌额": [0.3, 0.1],
|
||||
"换手率": [1.0, 1.2],
|
||||
}
|
||||
)
|
||||
|
||||
|
||||
def _make_bs_result(rows: list[list[str]]):
|
||||
cursor = {"idx": -1}
|
||||
|
||||
def next_row():
|
||||
cursor["idx"] += 1
|
||||
return cursor["idx"] < len(rows)
|
||||
|
||||
def get_row_data():
|
||||
return rows[cursor["idx"]]
|
||||
|
||||
return SimpleNamespace(error_code="0", error_msg="success", next=next_row, get_row_data=get_row_data)
|
||||
|
||||
|
||||
@contextmanager
|
||||
def _fake_bs_query(result):
|
||||
yield result
|
||||
|
||||
|
||||
def test_fetch_one_stock_converts_baostock_rows():
|
||||
"""单股 BaoStock 读取应转成可入库行。"""
|
||||
result = _make_bs_result([
|
||||
["2026-05-08", "10.0", "10.5", "9.9", "10.3", "100", "1030", "10.0", "3.0", "1.0"],
|
||||
["2026-05-09", "10.2", "10.6", "10.0", "10.4", "120", "1248", "10.3", "0.97", "1.2"],
|
||||
])
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(result)):
|
||||
rows = daily._fetch_one_stock("600000", "20260508", "20260509")
|
||||
|
||||
assert len(rows) == 2
|
||||
assert rows[0]["code"] == "600000"
|
||||
assert rows[0]["date"] == "2026-05-08"
|
||||
assert rows[0]["open"] == 10.0
|
||||
assert rows[0]["turnover"] == 1030
|
||||
|
||||
|
||||
def test_fetch_one_stock_returns_empty_when_no_data():
|
||||
"""BaoStock 返回空表时应返回空列表。"""
|
||||
result = _make_bs_result([])
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(result)):
|
||||
rows = daily._fetch_one_stock("600000", "20260508", "20260508")
|
||||
|
||||
assert rows == []
|
||||
|
||||
|
||||
def test_fetch_one_stock_skips_rows_without_volume():
|
||||
"""volume 为空的日线行不应进入 stock_daily。"""
|
||||
result = _make_bs_result([
|
||||
["2026-05-08", "10.0", "10.5", "9.9", "10.3", "", "1030", "10.0", "3.0", "1.0"],
|
||||
["2026-05-09", "10.2", "10.6", "10.0", "10.4", "120", "1248", "10.3", "0.97", "1.2"],
|
||||
])
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(result)):
|
||||
rows = daily._fetch_one_stock("600000", "20260508", "20260509")
|
||||
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["date"] == "2026-05-09"
|
||||
assert rows[0]["volume"] == 120
|
||||
|
||||
|
||||
def test_fetch_daily_uses_baostock_fetcher():
|
||||
"""fetch_daily 应走 BaoStock 抓取并落库。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
hist_rows = [
|
||||
["2026-05-08", "10.0", "10.5", "9.9", "10.3", "100", "1030", "10.0", "3.0", "1.0"],
|
||||
]
|
||||
result = _make_bs_result(hist_rows)
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(result)), \
|
||||
patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-08"]}), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert:
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260508")
|
||||
|
||||
mock_upsert.assert_called_once()
|
||||
|
||||
|
||||
def test_fetch_daily_does_not_mark_no_data_days_when_empty_rows():
|
||||
"""fetch_daily 遇到空结果时不应写入停牌/无数据日期。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
empty_result = _make_bs_result([])
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(empty_result)), \
|
||||
patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08", "2026-05-09"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-09"]}), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert:
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260509")
|
||||
|
||||
assert mock_upsert.call_count == 0
|
||||
|
||||
|
||||
def test_fetch_daily_patches_missing_days_from_partial_range():
|
||||
"""--daily 遇到范围结果缺日时，应按天补抓并写入完整数据。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
range_rows = [
|
||||
{"code": "600000", "date": "2026-05-08", "open": 10.0, "close": 10.3, "high": 10.5, "low": 9.9,
|
||||
"volume": 100, "turnover": 1030, "amplitude": 5.8, "pct_change": 3.0, "change": 0.3, "turnover_rate": 1.0},
|
||||
]
|
||||
single_day_rows = [
|
||||
{"code": "600000", "date": "2026-05-09", "open": 10.2, "close": 10.4, "high": 10.6, "low": 10.0,
|
||||
"volume": 120, "turnover": 1248, "amplitude": 5.9, "pct_change": 0.97, "change": 0.1, "turnover_rate": 1.2},
|
||||
]
|
||||
|
||||
def fake_fetch(code, gap_start, gap_end, *, retry=3, delay=0.1):
|
||||
if gap_start == "20260508" and gap_end == "20260509":
|
||||
return list(range_rows)
|
||||
if gap_start == "20260509" and gap_end == "20260509":
|
||||
return list(single_day_rows)
|
||||
raise AssertionError((code, gap_start, gap_end))
|
||||
|
||||
with patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08", "2026-05-09"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-09"]}), \
|
||||
patch.object(daily, "_fetch_one_stock_with_retry", side_effect=fake_fetch), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert:
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260509")
|
||||
|
||||
mock_upsert.assert_called_once()
|
||||
model_cls, rows, index_columns = mock_upsert.call_args.args
|
||||
assert model_cls.__tablename__ == "stock_daily"
|
||||
assert index_columns == ["code", "date"]
|
||||
assert [row["date"] for row in rows] == ["2026-05-08", "2026-05-09"]
|
||||
|
||||
|
||||
def test_fetch_daily_no_data_only_marks_partial_gap_days():
|
||||
"""--daily-no-data-only 遇到部分行情时应补记缺失日期。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
hist_rows = [
|
||||
["2026-05-08", "10.0", "10.5", "9.9", "10.3", "100", "1030", "10.0", "3.0", "1.0"],
|
||||
]
|
||||
result = _make_bs_result(hist_rows)
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(result)), \
|
||||
patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08", "2026-05-09"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-09"]}), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert:
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260509", no_data_only=True)
|
||||
|
||||
assert mock_upsert.call_count == 1
|
||||
model_cls, rows, index_columns = mock_upsert.call_args.args
|
||||
assert model_cls.__tablename__ == "stock_no_data"
|
||||
assert index_columns == ["code", "date"]
|
||||
assert [row["date"] for row in rows] == ["2026-05-09"]
|
||||
|
||||
|
||||
def test_fetch_daily_no_data_only_marks_empty_result_as_no_data():
|
||||
"""--daily-no-data-only 遇到空结果时仍应写入停牌/无数据日期。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
empty_result = _make_bs_result([])
|
||||
|
||||
with patch.object(daily, "bs_query", lambda *args, **kwargs: _fake_bs_query(empty_result)), \
|
||||
patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08", "2026-05-09"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-09"]}), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert:
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260509", no_data_only=True)
|
||||
|
||||
assert mock_upsert.call_count == 1
|
||||
model_cls, rows, index_columns = mock_upsert.call_args.args
|
||||
assert model_cls.__tablename__ == "stock_no_data"
|
||||
assert index_columns == ["code", "date"]
|
||||
assert [row["date"] for row in rows] == ["2026-05-08", "2026-05-09"]
|
||||
|
||||
|
||||
def test_fetch_daily_skips_unsupported_non_a_share_codes():
|
||||
"""fetch_daily 应在请求前过滤非沪深 A 股代码。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
|
||||
with patch.object(daily, "get_stock_codes", return_value=["600000", "920200"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-08"]}) as mock_analyze, \
|
||||
patch.object(daily, "_fetch_one_stock_with_retry", return_value=[]) as mock_fetch, \
|
||||
patch.object(daily, "batch_upsert"):
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260508")
|
||||
|
||||
mock_analyze.assert_called_once()
|
||||
assert mock_analyze.call_args.args[0] == ["600000"]
|
||||
mock_fetch.assert_called_once_with("600000", "20260508", "20260508", retry=3, delay=0.2)
|
||||
|
||||
|
||||
def test_fetch_daily_retries_transient_baostock_failure():
|
||||
"""fetch_daily 遇到临时断连时应重试。"""
|
||||
fake_session = SimpleNamespace(execute=lambda *args, **kwargs: [], close=lambda: None)
|
||||
hist_rows = [
|
||||
["2026-05-08", "10.0", "10.5", "9.9", "10.3", "100", "1030", "10.0", "3.0", "1.0"],
|
||||
]
|
||||
result = _make_bs_result(hist_rows)
|
||||
calls = {"count": 0}
|
||||
|
||||
def flaky_query(*args, **kwargs):
|
||||
calls["count"] += 1
|
||||
if calls["count"] == 1:
|
||||
raise ConnectionError("boom")
|
||||
return _fake_bs_query(result)
|
||||
|
||||
with patch.object(daily, "bs_query", flaky_query), \
|
||||
patch.object(daily, "get_fetch_config", return_value={"delay": 0, "retry": 2}), \
|
||||
patch.object(daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(daily, "get_trading_days", return_value=["2026-05-08"]), \
|
||||
patch.object(daily, "get_session", return_value=fake_session), \
|
||||
patch.object(daily, "_analyze_gaps", return_value={"600000": ["2026-05-08", "2026-05-08"]}), \
|
||||
patch.object(daily, "batch_upsert") as mock_upsert, \
|
||||
patch.object(daily.time, "sleep", return_value=None):
|
||||
daily.fetch_daily(start_date="20260508", end_date="20260508")
|
||||
|
||||
assert calls["count"] == 2
|
||||
mock_upsert.assert_called_once()
|
||||
|
||||
|
||||
def test_analyze_gaps_marks_incomplete_volume_as_gap():
|
||||
"""已存在但关键字段为空时，应视为缺口并重抓。"""
|
||||
class FakeSession:
|
||||
def __init__(self):
|
||||
self.calls = 0
|
||||
|
||||
def execute(self, *args, **kwargs):
|
||||
self.calls += 1
|
||||
return [("600000", "2026-05", 2, 1)]
|
||||
|
||||
def close(self):
|
||||
return None
|
||||
|
||||
fake_session = FakeSession()
|
||||
|
||||
with patch.object(daily, "get_ipo_dates", return_value={"600000": "2026-01-01"}), \
|
||||
patch.object(daily, "_get_no_data_dates", return_value={}), \
|
||||
patch.object(daily, "get_session", return_value=fake_session):
|
||||
gaps = daily._analyze_gaps(
|
||||
["600000"],
|
||||
"20260508",
|
||||
"20260509",
|
||||
["2026-05-08", "2026-05-09"],
|
||||
)
|
||||
|
||||
assert gaps["600000"] == ["2026-05-08", "2026-05-09"]
|
||||
assert fake_session.calls == 1
|
||||
|
||||
|
||||
def test_analyze_gaps_detects_missing_middle_trading_day():
|
||||
"""已有头尾数据但中间缺交易日时，也应纳入缺口。"""
|
||||
class FakeSession:
|
||||
def __init__(self):
|
||||
self.calls = 0
|
||||
|
||||
def execute(self, *args, **kwargs):
|
||||
self.calls += 1
|
||||
if self.calls == 1:
|
||||
return [("600000", "2026-05", 2, 0)]
|
||||
return [("2026-05-08",), ("2026-05-11",)]
|
||||
|
||||
def close(self):
|
||||
return None
|
||||
|
||||
fake_session = FakeSession()
|
||||
|
||||
with patch.object(daily, "get_ipo_dates", return_value={"600000": "2026-01-01"}), \
|
||||
patch.object(daily, "_get_no_data_dates", return_value={}), \
|
||||
patch.object(daily, "get_session", return_value=fake_session):
|
||||
gaps = daily._analyze_gaps(
|
||||
["600000"],
|
||||
"20260508",
|
||||
"20260511",
|
||||
["2026-05-08", "2026-05-09", "2026-05-11"],
|
||||
)
|
||||
|
||||
assert gaps["600000"] == ["2026-05-09", "2026-05-09"]
|
||||
assert fake_session.calls == 2
|
||||
@@ -0,0 +1,151 @@
|
||||
"""验证数据库表结构约束。"""
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.db import (
|
||||
IndustryNode,
|
||||
IndexMin5,
|
||||
StockDaily,
|
||||
StockF10Evidence,
|
||||
StockIndustry,
|
||||
StockTradeStatus,
|
||||
get_stock_codes,
|
||||
invalidate_stock_codes_cache,
|
||||
batch_upsert,
|
||||
get_stock_minute_model,
|
||||
normalize_minute_freq,
|
||||
)
|
||||
|
||||
|
||||
def test_stock_daily_volume_is_not_null():
|
||||
"""stock_daily 业务列应全部为非空列。"""
|
||||
required_columns = (
|
||||
"open",
|
||||
"close",
|
||||
"high",
|
||||
"low",
|
||||
"volume",
|
||||
"turnover",
|
||||
"amplitude",
|
||||
"pct_change",
|
||||
"change",
|
||||
"turnover_rate",
|
||||
)
|
||||
for name in required_columns:
|
||||
assert StockDaily.__table__.c[name].nullable is False
|
||||
|
||||
|
||||
def test_only_stock_min5_model_is_supported():
|
||||
assert get_stock_minute_model("5m").__tablename__ == "stock_min5"
|
||||
assert normalize_minute_freq("5分钟") == 5
|
||||
with pytest.raises(ValueError, match="仅支持 5m"):
|
||||
get_stock_minute_model("15m")
|
||||
|
||||
|
||||
def test_index_min5_uses_a_separate_table_from_stock_min5():
|
||||
assert IndexMin5.__tablename__ == "index_min5"
|
||||
assert "code" in IndexMin5.__table__.c
|
||||
assert "datetime" in IndexMin5.__table__.c
|
||||
|
||||
|
||||
def test_trade_status_model_has_status_fields():
|
||||
assert StockTradeStatus.__tablename__ == "stock_trade_status"
|
||||
for name in (
|
||||
"status",
|
||||
"source",
|
||||
"daily_exists",
|
||||
"min5_count",
|
||||
"reason",
|
||||
"checked_at",
|
||||
):
|
||||
assert name in StockTradeStatus.__table__.c
|
||||
|
||||
|
||||
def test_industry_models_use_single_parent_tree_and_current_membership():
|
||||
assert IndustryNode.__tablename__ == "industry_node"
|
||||
assert StockIndustry.__tablename__ == "stock_industry"
|
||||
assert "parent_code" in IndustryNode.__table__.c
|
||||
assert "industry_code" in StockIndustry.__table__.c
|
||||
assert StockF10Evidence.__tablename__ == "stock_f10_evidence"
|
||||
assert "evidence_text" in StockF10Evidence.__table__.c
|
||||
|
||||
|
||||
def test_batch_upsert_splits_large_inputs():
|
||||
"""批量 upsert 应按配置分块执行，避免单条 SQL 过大。"""
|
||||
calls = []
|
||||
fake_session = SimpleNamespace(
|
||||
get_bind=lambda: SimpleNamespace(dialect=SimpleNamespace(name="mysql")),
|
||||
execute=lambda stmt: calls.append(stmt),
|
||||
commit=lambda: calls.append("commit"),
|
||||
rollback=lambda: None,
|
||||
close=lambda: None,
|
||||
)
|
||||
rows = [
|
||||
{
|
||||
"code": f"60000{i}",
|
||||
"date": "2026-05-15",
|
||||
"open": 1,
|
||||
"close": 1,
|
||||
"high": 1,
|
||||
"low": 1,
|
||||
"volume": 1,
|
||||
"turnover": 1,
|
||||
"amplitude": 1,
|
||||
"pct_change": 1,
|
||||
"change": 1,
|
||||
"turnover_rate": 1,
|
||||
}
|
||||
for i in range(3)
|
||||
]
|
||||
|
||||
with patch("src.db.runtime.get_fetch_config", return_value={"db_batch_size": 2}), \
|
||||
patch("src.db.runtime.get_session", return_value=fake_session):
|
||||
batch_upsert(StockDaily, rows, ["code", "date"])
|
||||
|
||||
assert len([item for item in calls if item != "commit"]) == 2
|
||||
assert calls[-1] == "commit"
|
||||
|
||||
|
||||
def test_upsert_uses_callers_transaction_and_updates_explicit_server_default():
|
||||
from datetime import datetime
|
||||
from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
engine = create_engine("sqlite://")
|
||||
StockIndustry.__table__.create(engine)
|
||||
row = {"source": "test", "stock_code": "002371", "industry_code": "ETCH", "active": True, "updated_at": datetime(2026, 9, 25)}
|
||||
keys = ["source", "stock_code", "industry_code"]
|
||||
with Session(engine) as session:
|
||||
batch_upsert(StockIndustry, [row], keys, session=session)
|
||||
session.commit()
|
||||
batch_upsert(StockIndustry, [{**row, "active": False}], keys, session=session)
|
||||
assert session.scalar(select(StockIndustry.active)) is False
|
||||
session.rollback()
|
||||
assert session.scalar(select(StockIndustry.active)) is True
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def test_get_stock_codes_queries_in_code_order():
|
||||
"""全市场任务读取股票列表时应按代码稳定排序。"""
|
||||
captured = []
|
||||
fake_session = SimpleNamespace(
|
||||
execute=lambda statement: captured.append(str(statement)) or [],
|
||||
close=lambda: None,
|
||||
)
|
||||
|
||||
invalidate_stock_codes_cache()
|
||||
with patch("src.db.runtime.get_session", return_value=fake_session):
|
||||
get_stock_codes()
|
||||
|
||||
assert "ORDER BY stock_info.code" in captured[0]
|
||||
|
||||
|
||||
def test_db_package_re_exports_canonical_modules():
|
||||
from src import db
|
||||
from src.db import models, runtime
|
||||
|
||||
assert db.StockDaily is models.StockDaily
|
||||
assert db.batch_upsert is runtime.batch_upsert
|
||||
@@ -0,0 +1,31 @@
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.app.commands import build_parser, run_command, selected_commands
|
||||
from src.fetchers.index_daily import _index_bars_to_rows
|
||||
|
||||
|
||||
def test_index_bars_to_rows_calculates_pct_change():
|
||||
rows = _index_bars_to_rows(
|
||||
"000001",
|
||||
[
|
||||
{"datetime": "2026-07-09", "open": 100, "high": 102, "low": 99, "close": 100, "vol": 10, "amount": 20},
|
||||
{"datetime": "2026-07-10", "open": 100, "high": 103, "low": 100, "close": 102, "vol": 11, "amount": 22},
|
||||
],
|
||||
"20260709",
|
||||
"20260710",
|
||||
)
|
||||
|
||||
assert len(rows) == 2
|
||||
assert rows[1]["pct_change"] == 2.0
|
||||
assert rows[1]["amount"] == 22
|
||||
assert rows[1]["volume"] == 110_000
|
||||
|
||||
|
||||
def test_index_daily_command_dispatches_service():
|
||||
args = build_parser().parse_args(["--index-daily", "--start-date", "20260701"])
|
||||
assert selected_commands(args) == ["index_daily"]
|
||||
|
||||
with patch("src.services.index_service.sync_index_daily") as sync:
|
||||
run_command("index_daily", args)
|
||||
|
||||
sync.assert_called_once_with(start_date="20260701", end_date=None)
|
||||
@@ -0,0 +1,918 @@
|
||||
from datetime import datetime
|
||||
|
||||
import pandas as pd
|
||||
import pytest
|
||||
|
||||
import src.fetchers.industry_chain as industry_chain
|
||||
from src.fetchers.industry_chain import (
|
||||
CHAIN_SPECS,
|
||||
classify_chain_profile,
|
||||
classify_chain_profiles,
|
||||
classify_semiconductor_profile,
|
||||
parse_ths_concept_evidence_html,
|
||||
parse_ths_operate_profile_html,
|
||||
semiconductor_chain_nodes,
|
||||
)
|
||||
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def capture_membership_writes_with_existing_test_sink(monkeypatch):
|
||||
# Persistence semantics are exercised against a real SQL engine in test_industry_memberships.
|
||||
monkeypatch.setattr(
|
||||
industry_chain, "replace_stock_industry_memberships",
|
||||
lambda rows: industry_chain.batch_upsert(industry_chain.StockIndustry, rows, ["source", "stock_code", "industry_code"]),
|
||||
)
|
||||
|
||||
|
||||
def _profile(main_business: str, product_type: str = "", product_name: str = "", business_scope: str = ""):
|
||||
return pd.DataFrame([{
|
||||
"主营业务": main_business,
|
||||
"产品类型": product_type,
|
||||
"产品名称": product_name,
|
||||
"经营范围": business_scope,
|
||||
}])
|
||||
|
||||
|
||||
def _spec(key: str):
|
||||
return next(spec for spec in CHAIN_SPECS if spec.key == key)
|
||||
|
||||
|
||||
def test_semiconductor_chain_nodes_form_upstream_midstream_downstream_tree():
|
||||
rows = semiconductor_chain_nodes(now=datetime(2026, 7, 12, 9, 0))
|
||||
by_code = {row["industry_code"]: row for row in rows}
|
||||
|
||||
assert by_code["CHAIN_SEMI_UP"]["parent_code"] == "CHAIN_SEMI"
|
||||
assert by_code["CHAIN_SEMI_MID"]["parent_code"] == "CHAIN_SEMI"
|
||||
assert by_code["CHAIN_SEMI_DOWN"]["parent_code"] == "CHAIN_SEMI"
|
||||
assert by_code["CHAIN_SEMI_MID_DESIGN_MEMORY"]["parent_code"] == "CHAIN_SEMI_MID_DESIGN"
|
||||
|
||||
|
||||
def test_semiconductor_classifier_identifies_equipment_foundry_and_packaging():
|
||||
assert classify_semiconductor_profile(_profile("半导体基础产品", "电子工艺装备", "半导体设备、刻蚀机")) == "CHAIN_SEMI_UP_EQ_ETCHING"
|
||||
assert classify_semiconductor_profile(_profile("集成电路晶圆代工业务")) == "CHAIN_SEMI_MID_FOUNDRY"
|
||||
assert classify_semiconductor_profile(_profile("提供集成电路封装测试服务")) == "CHAIN_SEMI_MID_PACKAGE_TEST"
|
||||
|
||||
|
||||
def test_semiconductor_classifier_descends_chip_design_by_product():
|
||||
assert classify_semiconductor_profile(_profile("存储器、微控制器和模拟芯片研发", "存储芯片")) == "CHAIN_SEMI_MID_DESIGN_MEMORY"
|
||||
assert classify_semiconductor_profile(_profile("高性能模拟集成电路研究与销售", "信号链集成电路")) == "CHAIN_SEMI_MID_DESIGN_ANALOG"
|
||||
|
||||
|
||||
def test_semiconductor_classifier_uses_generic_leaf_without_product_evidence():
|
||||
assert classify_semiconductor_profile(_profile("半导体产品研发与销售")) == "CHAIN_SEMI_MID_OTHER"
|
||||
assert classify_chain_profile(
|
||||
_profile("城市燃气供应与销售"),
|
||||
_spec("半导体"),
|
||||
allow_default=False,
|
||||
) is None
|
||||
|
||||
|
||||
def test_product_classifier_keeps_sibling_memberships_without_generic_parent():
|
||||
codes = classify_chain_profiles(
|
||||
_profile("半导体设备研发生产", "半导体设备", "刻蚀设备、PVD设备、ALD设备、清洗设备"),
|
||||
_spec("半导体"),
|
||||
)
|
||||
assert set(codes) == {"CHAIN_SEMI_UP_EQ_ETCHING", "CHAIN_SEMI_UP_EQ_DEPOSITION", "CHAIN_SEMI_UP_EQ_CLEAN"}
|
||||
assert len(codes) == 3
|
||||
|
||||
|
||||
@pytest.mark.parametrize(("chain", "business", "expected"), [
|
||||
("种植业", "玉米种子、小麦种子研发生产", {"CHAIN_CROP_UP_SEED_CORN", "CHAIN_CROP_UP_SEED_OTHER"}),
|
||||
("有色金属", "铜精矿、锑制品生产", {"CHAIN_METAL_COPPER_MINE", "CHAIN_METAL_RARE_OTHER"}),
|
||||
("医疗服务", "综合性医院、口腔医院运营", {"CHAIN_CARE_MID_GENERAL", "CHAIN_CARE_MID_DENTAL"}),
|
||||
])
|
||||
def test_multi_membership_keeps_real_businesses_with_generic_code_suffix(chain, business, expected):
|
||||
assert set(classify_chain_profiles(_profile(business), _spec(chain))) == expected
|
||||
|
||||
|
||||
def test_generic_design_fallback_only_yields_to_a_specific_design_product():
|
||||
spec = _spec("半导体")
|
||||
assert set(classify_chain_profiles(_profile("集成电路设计及刻蚀设备研发生产"), spec)) == {
|
||||
"CHAIN_SEMI_MID_DESIGN_OTHER", "CHAIN_SEMI_UP_EQ_ETCHING",
|
||||
}
|
||||
assert classify_chain_profiles(_profile("集成电路设计及存储芯片研发生产"), spec) == ["CHAIN_SEMI_MID_DESIGN_MEMORY"]
|
||||
|
||||
|
||||
def test_taxonomy_refresh_keeps_legacy_membership_visible_after_fetch_failure(monkeypatch):
|
||||
from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
engine = create_engine("sqlite://")
|
||||
industry_chain.IndustryNode.__table__.create(engine)
|
||||
industry_chain.StockIndustry.__table__.create(engine)
|
||||
old_code = "CHAIN_SEMI_UP_EQ_ETCH"
|
||||
now = datetime(2026, 9, 25)
|
||||
with Session(engine) as session:
|
||||
session.add(industry_chain.IndustryNode(source="industry_chain", industry_code=old_code, industry_name="刻蚀与薄膜沉积", parent_code="CHAIN_SEMI_UP_EQUIPMENT", industry_level=4, active=True, updated_at=now))
|
||||
session.add(industry_chain.StockIndustry(source="industry_chain", stock_code="002371", industry_code=old_code, updated_at=now))
|
||||
session.commit()
|
||||
|
||||
def upsert_nodes(model, rows, _keys):
|
||||
assert model is industry_chain.IndustryNode
|
||||
with Session(engine) as session:
|
||||
for row in rows:
|
||||
node = session.scalar(select(model).where(model.source == row["source"], model.industry_code == row["industry_code"]))
|
||||
if node is None:
|
||||
session.add(model(**row))
|
||||
else:
|
||||
for key, value in row.items():
|
||||
setattr(node, key, value)
|
||||
session.commit()
|
||||
|
||||
monkeypatch.setattr(industry_chain, "get_session", lambda: Session(engine))
|
||||
monkeypatch.setattr(industry_chain, "batch_upsert", upsert_nodes)
|
||||
monkeypatch.setattr(industry_chain, "_refresh_ths_stock_chain_evidence", lambda *_: (_ for _ in ()).throw(RuntimeError("unavailable")))
|
||||
spec = _spec("半导体")
|
||||
industry_chain._activate_nodes(spec, now)
|
||||
assert industry_chain._sync_chain(spec, symbol="002371", limit=None, refresh=True, workers=1, evidence_cache={}) == (0, 0, 1)
|
||||
with Session(engine) as session:
|
||||
visible = session.execute(
|
||||
select(industry_chain.StockIndustry.stock_code)
|
||||
.join(industry_chain.IndustryNode, (industry_chain.IndustryNode.source == industry_chain.StockIndustry.source) & (industry_chain.IndustryNode.industry_code == industry_chain.StockIndustry.industry_code))
|
||||
.where(industry_chain.IndustryNode.active.is_(True))
|
||||
).scalars().all()
|
||||
assert visible == ["002371"]
|
||||
assert all(code != old_code for code, _ in spec.rules)
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def test_multi_membership_keeps_business_evidence_filters():
|
||||
profile = _profile("刻蚀设备研发生产；拟布局薄膜沉积设备；产品应用于清洗设备")
|
||||
assert classify_chain_profiles(profile, _spec("半导体"), "公司研发薄膜沉积设备") == ["CHAIN_SEMI_UP_EQ_ETCHING"]
|
||||
assert classify_chain_profiles(_profile("城市燃气供应"), _spec("半导体"), allow_default=False) == []
|
||||
|
||||
|
||||
def test_sync_writes_all_memberships_counts_stocks_and_preserves_failed_fetch(monkeypatch):
|
||||
written = []
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda _: {"002371", "688012"})
|
||||
monkeypatch.setattr(industry_chain, "batch_upsert", lambda _model, _rows, _keys: None)
|
||||
monkeypatch.setattr(industry_chain, "replace_stock_industry_memberships", written.extend)
|
||||
monkeypatch.setattr(industry_chain, "_refresh_ths_stock_chain_evidence", lambda *_: (_ for _ in ()).throw(RuntimeError("unavailable")))
|
||||
result = industry_chain._sync_chain(
|
||||
_spec("半导体"), symbol="002371,688012,600000", limit=None, refresh=False, workers=1,
|
||||
evidence_cache={"002371": "【F10主营介绍】主营业务：刻蚀设备、薄膜沉积设备研发生产", "600000": "【F10主营介绍】主营业务：银行业务"},
|
||||
)
|
||||
assert result == (1, 1, 1)
|
||||
assert {row["industry_code"] for row in written if row["stock_code"] == "002371"} == {"CHAIN_SEMI_UP_EQ_ETCHING", "CHAIN_SEMI_UP_EQ_DEPOSITION"}
|
||||
assert [row["industry_code"] for row in written if row["stock_code"] == "600000"] == ["CHAIN_SEMI_UNMATCHED"]
|
||||
assert not any(row["stock_code"] == "688012" for row in written)
|
||||
|
||||
|
||||
def test_active_classifier_writes_multiple_nodes_for_same_source(monkeypatch):
|
||||
written = []
|
||||
monkeypatch.setattr(industry_chain, "_unclassified_active_candidates", lambda *_: [{"code": "002371"}])
|
||||
monkeypatch.setattr(industry_chain, "_load_f10_evidence_cache", lambda: {"002371": "【F10主营介绍】主营业务：刻蚀设备、薄膜沉积设备研发生产"})
|
||||
monkeypatch.setattr(industry_chain, "_activate_nodes", lambda *_: 0)
|
||||
monkeypatch.setattr(industry_chain, "replace_stock_industry_memberships", written.extend)
|
||||
assert industry_chain.classify_unclassified_active(workers=1) == (1, 0, 0)
|
||||
assert {row["industry_code"] for row in written if row["source"] == "industry_chain"} == {"CHAIN_SEMI_UP_EQ_ETCHING", "CHAIN_SEMI_UP_EQ_DEPOSITION"}
|
||||
|
||||
|
||||
def test_each_chain_uses_an_independent_source_for_multi_chain_membership():
|
||||
sources = [spec.source for spec in CHAIN_SPECS]
|
||||
|
||||
assert len(sources) == len(set(sources))
|
||||
assert all(len(source) <= 20 for source in sources)
|
||||
|
||||
|
||||
def test_sync_activates_curated_trees_and_attaches_each_chain(monkeypatch):
|
||||
class FakeSession:
|
||||
def execute(self, *_args, **_kwargs):
|
||||
return None
|
||||
|
||||
def scalars(self, *_args, **_kwargs):
|
||||
return []
|
||||
|
||||
def commit(self):
|
||||
return None
|
||||
|
||||
def close(self):
|
||||
return None
|
||||
|
||||
activated = []
|
||||
synced = []
|
||||
shared_cache = {"600000": "cached evidence"}
|
||||
monkeypatch.setattr(industry_chain, "get_session", lambda: FakeSession())
|
||||
monkeypatch.setattr(industry_chain, "_load_f10_evidence_cache", lambda: shared_cache)
|
||||
monkeypatch.setattr(industry_chain, "_activate_nodes", lambda spec, _now: activated.append(spec.key) or len(spec.definitions))
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"_sync_chain",
|
||||
lambda spec, **kwargs: synced.append((spec.key, kwargs["evidence_cache"])) or (1, 0, 0),
|
||||
)
|
||||
|
||||
node_count, matched, failed = industry_chain.sync_industry_chains(refresh=False)
|
||||
|
||||
assert node_count == sum(len(spec.definitions) for spec in CHAIN_SPECS)
|
||||
assert matched == len(CHAIN_SPECS)
|
||||
assert failed == 0
|
||||
assert activated == [spec.key for spec in CHAIN_SPECS]
|
||||
assert [key for key, _ in synced] == activated
|
||||
assert all(cache is shared_cache for _, cache in synced)
|
||||
|
||||
|
||||
def test_sync_deactivates_obsolete_nodes_without_removing_history(monkeypatch):
|
||||
from sqlalchemy import create_engine, select
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
engine = create_engine("sqlite://")
|
||||
industry_chain.IndustryNode.__table__.create(engine)
|
||||
industry_chain.StockIndustry.__table__.create(engine)
|
||||
now = datetime(2026, 9, 25)
|
||||
obsolete = "industry_chain_old"
|
||||
with Session(engine) as session:
|
||||
for source in (obsolete, CHAIN_SPECS[0].source):
|
||||
session.add(industry_chain.IndustryNode(source=source, industry_code="NODE", industry_name="Node", industry_level=1, active=True, updated_at=now))
|
||||
session.add(industry_chain.StockIndustry(source=source, stock_code="002371", industry_code="NODE", active=True, updated_at=now))
|
||||
session.commit()
|
||||
deactivated_sources = []
|
||||
monkeypatch.setattr(industry_chain, "get_session", lambda: Session(engine))
|
||||
monkeypatch.setattr(industry_chain, "_activate_nodes", lambda *_: 0)
|
||||
monkeypatch.setattr(industry_chain, "deactivate_stock_industry_sources", deactivated_sources.extend)
|
||||
assert industry_chain.sync_industry_chains(taxonomy_only=True) == (0, 0, 0)
|
||||
with Session(engine) as session:
|
||||
nodes = dict(session.execute(select(industry_chain.IndustryNode.source, industry_chain.IndustryNode.active)).all())
|
||||
assert nodes == {obsolete: False, CHAIN_SPECS[0].source: True}
|
||||
assert len(session.scalars(select(industry_chain.StockIndustry.id)).all()) == 2
|
||||
assert deactivated_sources == [obsolete]
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def test_existing_chain_codes_ignore_inactive_history(monkeypatch):
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
engine = create_engine("sqlite://")
|
||||
industry_chain.StockIndustry.__table__.create(engine)
|
||||
source = CHAIN_SPECS[0].source
|
||||
with Session(engine) as session:
|
||||
for code, active in (("002371", True), ("688012", False)):
|
||||
session.add(industry_chain.StockIndustry(source=source, stock_code=code, industry_code="NODE", active=active, updated_at=datetime(2026, 9, 25)))
|
||||
session.commit()
|
||||
monkeypatch.setattr(industry_chain, "get_session", lambda: Session(engine))
|
||||
assert industry_chain._existing_chain_codes(source) == {"002371"}
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def test_sync_chain_explicit_symbol_enters_candidate_pool(monkeypatch):
|
||||
written = []
|
||||
aidc = _spec("AIDC")
|
||||
evidence = (
|
||||
"【F10主营介绍】主营业务：数据中心温控系统研发生产； "
|
||||
"产品类型：温控系统； 产品名称：数据中心温控系统； 经营范围："
|
||||
)
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_args: [])
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda _source: set())
|
||||
monkeypatch.setattr(industry_chain, "_refresh_ths_stock_chain_evidence", lambda code, _old: evidence)
|
||||
monkeypatch.setattr(industry_chain, "batch_upsert", lambda _model, rows, _keys: written.extend(rows))
|
||||
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
aidc,
|
||||
symbol="000811",
|
||||
limit=None,
|
||||
refresh=False,
|
||||
workers=1,
|
||||
evidence_cache={},
|
||||
)
|
||||
|
||||
memberships = [row for row in written if row.get("source") == aidc.source]
|
||||
assert (matched, unmatched, failed) == (1, 0, 0)
|
||||
assert memberships[-1]["stock_code"] == "000811"
|
||||
assert memberships[-1]["industry_code"] == "CHAIN_AIDC_INFRA_COOL"
|
||||
|
||||
|
||||
def test_robot_space_and_cpo_profiles_descend_to_product_leaves():
|
||||
robot, space, cpo = CHAIN_SPECS[1:4]
|
||||
|
||||
assert classify_chain_profile(_profile("谐波减速器研发和制造"), robot) == "CHAIN_ROBOT_UP_REDUCER"
|
||||
assert classify_chain_profile(_profile("商业卫星整星和卫星载荷研制"), space) == "CHAIN_SPACE_MID_SAT_PAYLOAD"
|
||||
assert classify_chain_profile(_profile("高速光模块及CPO光引擎研发"), cpo) == "CHAIN_CPO_MID_ENGINE"
|
||||
|
||||
|
||||
def test_semiconductor_display_profile_maps_to_panel_manufacturing():
|
||||
semiconductor = CHAIN_SPECS[0]
|
||||
|
||||
assert classify_chain_profile(_profile("显示器件业务、TFT-LCD和AMOLED显示面板制造"), semiconductor) == "CHAIN_SEMI_MID_DISPLAY"
|
||||
|
||||
|
||||
def test_china_spacesat_business_maps_to_satellite_manufacturing():
|
||||
space = CHAIN_SPECS[2]
|
||||
profile = _profile(
|
||||
"卫星及相关产品的研发、设计、制造、销售；航天技术应用及相关产品研发。",
|
||||
"宇航制造业务、卫星应用业务",
|
||||
"宇航制造业务、卫星应用技术设备、卫星应用服务",
|
||||
)
|
||||
|
||||
assert classify_chain_profile(profile, space) == "CHAIN_SPACE_MID_SAT_ASSEMBLY"
|
||||
|
||||
|
||||
def test_thematic_chain_rejects_candidate_without_main_business_evidence():
|
||||
assert classify_chain_profile(_profile("房地产开发与销售"), CHAIN_SPECS[1]) is None
|
||||
|
||||
|
||||
def test_solid_state_battery_profiles_cover_material_cell_and_application():
|
||||
solid_state_battery = CHAIN_SPECS[4]
|
||||
|
||||
assert classify_chain_profile(_profile("硫化物固态电解质研发生产"), solid_state_battery) == "CHAIN_SSB_UP_ELEC_SULFIDE"
|
||||
assert classify_chain_profile(_profile("全固态电池和半固态电芯制造"), solid_state_battery) == "CHAIN_SSB_MID_CELL_FULL"
|
||||
assert classify_chain_profile(_profile("新能源汽车整车研发与销售"), solid_state_battery) == "CHAIN_SSB_DOWN_AUTO"
|
||||
|
||||
|
||||
def test_f10_generic_concept_names_do_not_force_a_leaf_assignment():
|
||||
cpo = CHAIN_SPECS[3]
|
||||
solid_state_battery = CHAIN_SPECS[4]
|
||||
|
||||
assert classify_chain_profile(None, cpo, "CPO概念") is None
|
||||
assert classify_chain_profile(None, cpo, "公司生产800G高速光模块") == "CHAIN_CPO_MID_MODULE_PLUG"
|
||||
assert classify_chain_profile(None, solid_state_battery, "固态电池概念") is None
|
||||
assert classify_chain_profile(None, solid_state_battery, "硫化物固态电解质研发") == "CHAIN_SSB_UP_ELEC_SULFIDE"
|
||||
|
||||
|
||||
def test_cpo_report_terms_descend_to_electrical_chip_leaves():
|
||||
cpo = CHAIN_SPECS[3]
|
||||
|
||||
assert classify_chain_profile(_profile("高速光模块O-DSP芯片研发"), cpo) == "CHAIN_CPO_UP_ELECTRIC_DSP"
|
||||
assert classify_chain_profile(_profile("光模块TIA芯片和Laser Driver研发"), cpo) == "CHAIN_CPO_UP_ELECTRIC_ANALOG"
|
||||
|
||||
|
||||
def test_cpo_profile_classifies_high_speed_pcb_manufacturer():
|
||||
cpo = CHAIN_SPECS[3]
|
||||
|
||||
assert classify_chain_profile(_profile("印制电路板的研发、生产和销售"), cpo) == "CHAIN_CPO_UP_PCB"
|
||||
assert classify_chain_profile(_profile("高端电子级玻纤布和低介电电子布研发生产"), cpo) == "CHAIN_CPO_UP_PCB"
|
||||
|
||||
|
||||
def test_ths_f10_decoder_supports_utf8_and_gbk():
|
||||
html = "<div>主营业务：印制电路板；产品名称：高速PCB</div>"
|
||||
|
||||
assert "印制电路板" in industry_chain._decode_ths_f10_html(html.encode("utf-8"))
|
||||
assert "印制电路板" in industry_chain._decode_ths_f10_html(html.encode("gbk"))
|
||||
|
||||
|
||||
def test_f10_profile_parser_accepts_main_business_without_following_fields():
|
||||
evidence = f"概念说明 {industry_chain.F10_PROFILE_MARKER}主营业务：互联网游戏的研发和运营。"
|
||||
|
||||
concept, fields = industry_chain._split_f10_profile_evidence(evidence)
|
||||
|
||||
assert concept == "概念说明 "
|
||||
assert fields[0] == "互联网游戏的研发和运营。"
|
||||
|
||||
|
||||
def test_new_report_chains_descend_to_specific_product_leaves():
|
||||
assert classify_chain_profile(_profile("eVTOL电动垂直起降飞行器整机研发"), _spec("低空经济")) == "CHAIN_LA_MID_EVTOL"
|
||||
assert classify_chain_profile(_profile("数据中心冷板液冷和CDU产品"), _spec("AIDC")) == "CHAIN_AIDC_INFRA_COOL_COLD"
|
||||
assert classify_chain_profile(_profile("智能终端的结构件、功能模组、整机组装等配套服务"), _spec("消费电子")) == "CHAIN_CE_MID_ODM"
|
||||
assert classify_chain_profile(_profile("人形机器人等智能终端的结构件研发生产"), _spec("机器人")) == "CHAIN_ROBOT_UP_MATERIAL"
|
||||
assert classify_chain_profile(_profile("显示器件业务、MLED业务和传感业务"), _spec("消费电子")) == "CHAIN_CE_UP_DISPLAY"
|
||||
assert classify_chain_profile(_profile("设计研发制造通信及移动网络设备、云计算电子设备产品"), _spec("AIDC")) == "CHAIN_AIDC_UP_NETWORK"
|
||||
assert classify_chain_profile(_profile("全钒液流电池储能系统"), _spec("储能")) == "CHAIN_ES_MID_FLOW"
|
||||
assert classify_chain_profile(_profile("HJT异质结电池研发制造"), _spec("光伏")) == "CHAIN_PV_MID_CELL_HJT"
|
||||
assert classify_chain_profile(_profile("海上风机整机研发制造"), _spec("风电")) == "CHAIN_WIND_MID_OFFSHORE"
|
||||
assert classify_chain_profile(_profile("PEM电解槽与质子交换膜电解制氢设备"), _spec("氢能源")) == "CHAIN_H2_UP_PEM"
|
||||
assert classify_chain_profile(_profile("抗体偶联药物ADC研发"), _spec("创新药")) == "CHAIN_DRUG_MID_ADC"
|
||||
assert classify_chain_profile(_profile("腔镜手术机器人研发制造"), _spec("医疗器械")) == "CHAIN_MED_MID_ROBOT"
|
||||
|
||||
|
||||
def test_audited_business_synonyms_map_to_expected_chains():
|
||||
assert classify_chain_profile(_profile("数据中心温控系统研发生产"), _spec("AIDC")) == "CHAIN_AIDC_INFRA_COOL"
|
||||
assert classify_chain_profile(_profile("AI营销、AI应用分发和空间智能MaaS平台"), _spec("AIDC")) == "CHAIN_AIDC_DOWN_ENTERPRISE"
|
||||
assert classify_chain_profile(_profile("固态电池电解质材料研发生产"), _spec("固态电池")) == "CHAIN_SSB_UP_ELECTROLYTE"
|
||||
assert classify_chain_profile(_profile("工业尾气提纯高纯氢气生产"), _spec("氢能源")) == "CHAIN_H2_UP_INDUSTRIAL"
|
||||
assert classify_chain_profile(_profile("新能源汽车内饰用皮革生产"), _spec("新能源汽车")) == "CHAIN_AUTO_UP_INTERIOR"
|
||||
assert classify_chain_profile(_profile("高分子材料防老化助剂和受阻胺光稳定剂生产"), _spec("石化精细化工")) == "CHAIN_CHEM_FINE_ADDITIVE"
|
||||
assert classify_chain_profile(_profile("数据中心工程总承包"), _spec("基础设施工程")) == "CHAIN_INFRA_MID_CONTRACT"
|
||||
|
||||
|
||||
def test_audited_rules_reject_broad_false_positive_terms():
|
||||
assert classify_chain_profile(_profile("城市燃气供应与销售"), _spec("煤化工")) is None
|
||||
assert classify_chain_profile(_profile("锂电池用电解铜箔生产"), _spec("电子元件")) is None
|
||||
assert classify_chain_profile(_profile("主焦煤供应链贸易服务"), _spec("煤化工")) == "CHAIN_COAL_DOWN_LOGISTICS"
|
||||
|
||||
|
||||
def test_f10_profile_business_outweighs_repeated_concept_text():
|
||||
evidence = (
|
||||
"存储芯片 存储芯片 存储芯片 存储芯片 "
|
||||
"【F10主营介绍】主营业务：半导体封装测试业务； "
|
||||
"产品类型：封装测试； 产品名称：封装测试； 经营范围："
|
||||
)
|
||||
|
||||
assert classify_chain_profile(None, _spec("半导体"), evidence) == "CHAIN_SEMI_MID_PACKAGE"
|
||||
|
||||
|
||||
def test_latest_report_chains_descend_to_verified_business_leaves():
|
||||
assert classify_chain_profile(_profile("新能源汽车三合一电驱动总成研发制造"), _spec("新能源汽车")) == "CHAIN_AUTO_UP_DRIVE_REDUCER"
|
||||
assert classify_chain_profile(_profile("相控阵雷达射频组件和雷达探测系统"), _spec("军工电子")) == "CHAIN_MIL_MID_RADAR"
|
||||
assert classify_chain_profile(_profile("AI眼镜整机和智能眼镜整机研发"), _spec("消费电子")) == "CHAIN_CE_DOWN_GLASSES"
|
||||
assert classify_chain_profile(_profile("国产分布式数据库管理系统"), _spec("软件云安全")) == "CHAIN_SW_UP_DB"
|
||||
assert classify_chain_profile(_profile("湿电子化学品和半导体化学品"), _spec("石化精细化工")) == "CHAIN_CHEM_FINE_ELECTRONIC"
|
||||
|
||||
|
||||
def test_pharma_healthcare_and_food_report_chains_use_business_leaves():
|
||||
assert classify_chain_profile(_profile("吸入制剂和缓控释复杂制剂研发生产"), _spec("化学制药")) == "CHAIN_PHARM_MID_COMPLEX"
|
||||
assert classify_chain_profile(_profile("中药配方颗粒和单味配方颗粒生产"), _spec("中药")) == "CHAIN_TCM_MID_GRANULE"
|
||||
assert classify_chain_profile(_profile("连锁眼科医院及眼科诊疗服务"), _spec("医疗服务")) == "CHAIN_CARE_MID_EYE"
|
||||
assert classify_chain_profile(_profile("DTP药房和特药配送服务"), _spec("医药流通")) == "CHAIN_DIST_MID_DTP"
|
||||
assert classify_chain_profile(_profile("速冻食品和冷冻调理食品生产"), _spec("食品饮料")) == "CHAIN_FOOD_MID_FROZEN"
|
||||
|
||||
|
||||
def test_new_healthcare_chains_keep_business_boundaries():
|
||||
assert classify_chain_profile(_profile("特色原料药生产"), _spec("创新药")) is None
|
||||
assert classify_chain_profile(_profile("特色原料药生产"), _spec("化学制药")) == "CHAIN_PHARM_UP_API_SPECIAL"
|
||||
assert classify_chain_profile(_profile("连锁药店和零售药房"), _spec("医疗服务")) is None
|
||||
assert classify_chain_profile(_profile("连锁药店和零售药房"), _spec("医药流通")) == "CHAIN_DIST_DOWN_CHAIN"
|
||||
|
||||
|
||||
def test_tourism_retail_and_agriculture_chains_use_specific_business_leaves():
|
||||
assert classify_chain_profile(_profile("中高端连锁酒店和度假酒店运营"), _spec("旅游酒店")) == "CHAIN_TOUR_MID_HOTEL_HIGH"
|
||||
assert classify_chain_profile(_profile("跨境电商平台和海外电商销售"), _spec("零售商业")) == "CHAIN_SHOP_DOWN_CROSS"
|
||||
assert classify_chain_profile(_profile("转基因玉米生物育种研发"), _spec("种植业")) == "CHAIN_CROP_UP_BIOTECH"
|
||||
assert classify_chain_profile(_profile("白羽肉鸡养殖和禽类屠宰"), _spec("畜禽养殖")) == "CHAIN_ANI_MID_POULTRY"
|
||||
assert classify_chain_profile(_profile("海参苗种繁育和海水养殖"), _spec("水产渔业")) == "CHAIN_AQUA_UP_SEED"
|
||||
assert classify_chain_profile(_profile("犬猫宠物疫苗研发生产"), _spec("宠物")) == "CHAIN_PET_UP_VACCINE"
|
||||
|
||||
|
||||
def test_agriculture_and_service_chains_keep_report_boundaries():
|
||||
assert classify_chain_profile(_profile("在线旅游平台和酒店预订"), _spec("零售商业")) is None
|
||||
assert classify_chain_profile(_profile("在线旅游平台和酒店预订"), _spec("旅游酒店")) == "CHAIN_TOUR_DOWN_OTA"
|
||||
assert classify_chain_profile(_profile("水产饲料和虾饲料"), _spec("畜禽养殖")) is None
|
||||
assert classify_chain_profile(_profile("水产饲料和虾饲料"), _spec("水产渔业")) == "CHAIN_AQUA_UP_FEED"
|
||||
assert classify_chain_profile(_profile("宠物医院和宠物医疗"), _spec("医疗服务")) is None
|
||||
assert classify_chain_profile(_profile("连锁宠物医院运营和宠物诊疗服务"), _spec("宠物")) == "CHAIN_PET_DOWN_HOSPITAL"
|
||||
assert classify_chain_profile(_profile("超声设备用于宠物医院"), _spec("宠物")) is None
|
||||
|
||||
|
||||
def test_home_textile_light_education_and_hr_chains_use_business_leaves():
|
||||
assert classify_chain_profile(_profile("扫地机器人和洗地机研发生产"), _spec("家电")) == "CHAIN_HOME_MID_CLEAN"
|
||||
assert classify_chain_profile(_profile("功能性针织面料和纺织印染加工"), _spec("纺织服装")) == "CHAIN_TEX_MID_DYE"
|
||||
assert classify_chain_profile(_profile("瓦楞纸箱和彩盒包装生产"), _spec("轻工家居")) == "CHAIN_LIGHT_MID_PACKAGE_PAPER"
|
||||
assert classify_chain_profile(_profile("公务员和事业单位招录培训"), _spec("教育")) == "CHAIN_EDU_MID_EXAM"
|
||||
assert classify_chain_profile(_profile("灵活用工平台与弹性用工服务"), _spec("人力资源服务")) == "CHAIN_HR_MID_FLEX"
|
||||
assert classify_chain_profile(_profile("客户端游戏与移动游戏研发运营"), _spec("传媒内容")) == "CHAIN_MEDIA_MID_GAME_FULL"
|
||||
assert classify_chain_profile(_profile("图书出版、数字出版和出版物发行"), _spec("传媒内容")) == "CHAIN_MEDIA_MID_PUBLISH"
|
||||
assert classify_chain_profile(_profile("原油和成品油轮运输业务"), _spec("交通物流")) == "CHAIN_TRANS_MID_SHIP_TANKER"
|
||||
assert classify_chain_profile(_profile("全国快递网络和快递服务"), _spec("交通物流")) == "CHAIN_TRANS_MID_EXPRESS"
|
||||
assert classify_chain_profile(_profile("燃煤火力发电和热电联产"), _spec("公用事业")) == "CHAIN_UTIL_MID_POWER_THERMAL"
|
||||
assert classify_chain_profile(_profile("城市燃气管网运营和天然气销售"), _spec("公用事业")) == "CHAIN_UTIL_MID_GAS_CITY"
|
||||
assert classify_chain_profile(_profile("生活垃圾焚烧发电项目运营"), _spec("环保")) == "CHAIN_ENV_MID_WASTE_INCIN"
|
||||
assert classify_chain_profile(_profile("环境监测仪器和污染源在线监测系统"), _spec("环保")) == "CHAIN_ENV_UP_MONITOR"
|
||||
|
||||
|
||||
def test_real_estate_material_steel_metal_and_finance_chains_use_business_leaves():
|
||||
assert classify_chain_profile(_profile("商品住宅开发与住宅项目销售"), _spec("房地产")) == "CHAIN_RE_MID_RESIDENTIAL"
|
||||
assert classify_chain_profile(_profile("产业园区开发运营和工业园区招商"), _spec("房地产")) == "CHAIN_RE_MID_PARK"
|
||||
assert classify_chain_profile(_profile("玻璃纤维生产及电子级玻纤布制造"), _spec("建筑材料")) == "CHAIN_MAT_MID_FIBER"
|
||||
assert classify_chain_profile(_profile("防水卷材和建筑防水材料生产"), _spec("建筑材料")) == "CHAIN_MAT_DOWN_WATERPROOF"
|
||||
assert classify_chain_profile(_profile("热轧板卷、冷轧板卷和汽车板生产"), _spec("钢铁")) == "CHAIN_STEEL_MID_FLAT"
|
||||
assert classify_chain_profile(_profile("特殊钢和轴承钢生产"), _spec("钢铁")) == "CHAIN_STEEL_MID_SPECIAL"
|
||||
assert classify_chain_profile(_profile("铜矿采选、阴极铜冶炼和电解铜生产"), _spec("有色金属")) == "CHAIN_METAL_COPPER_MINE"
|
||||
assert classify_chain_profile(_profile("锂辉石精矿和碳酸锂生产"), _spec("有色金属")) == "CHAIN_METAL_ENERGY_LI"
|
||||
assert classify_chain_profile(_profile("证券经纪业务和财富管理业务"), _spec("金融服务")) == "CHAIN_FIN_SEC_BROKER"
|
||||
assert classify_chain_profile(_profile("融资租赁业务和金融租赁业务"), _spec("金融服务")) == "CHAIN_FIN_MULTI_LEASE"
|
||||
assert classify_chain_profile(_profile("房地产开发和物业服务"), _spec("房地产")) == "CHAIN_RE_MID_INTEGRATED"
|
||||
assert classify_chain_profile(_profile("水泥、建材产品生产", "水泥、水泥制品", "水泥、纸面石膏板"), _spec("建筑材料")) == "CHAIN_MAT_MID_CEMENT"
|
||||
assert classify_chain_profile(_profile("预应力钢筒混凝土管和钢筋混凝土排水管生产", product_name="混凝土外加剂"), _spec("建筑材料")) == "CHAIN_MAT_MID_CONCRETE"
|
||||
assert classify_chain_profile(_profile("弹簧扁钢、优线和螺纹钢生产", product_name="汽车板簧"), _spec("钢铁")) == "CHAIN_STEEL_MID_SPECIAL"
|
||||
assert classify_chain_profile(_profile("商业银行及相关金融服务业务", product_name="投资银行业务"), _spec("金融服务")) == "CHAIN_FIN_BANK_STATE"
|
||||
assert classify_chain_profile(_profile("各类风险保障和资产管理", product_name="人身保险产品和服务、财产保险产品和服务"), _spec("金融服务")) == "CHAIN_FIN_INS_GROUP"
|
||||
assert classify_chain_profile(_profile("提供多元化的金融产品及服务", product_name="寿险及健康险业务、财产保险业务、银行业务、资产管理业务"), _spec("金融服务")) == "CHAIN_FIN_INS_GROUP"
|
||||
assert classify_chain_profile(_profile("期货经纪业务、财富管理业务、风险管理业务"), _spec("金融服务")) == "CHAIN_FIN_SEC_FUTURES"
|
||||
assert classify_chain_profile(_profile("钢铁与钢铁主业相关的加工配送业务", product_name="热轧板卷、钢管产品"), _spec("钢铁")) == "CHAIN_STEEL_MID_INTEGRATED"
|
||||
assert classify_chain_profile(_profile("加工、生产和销售离心球墨铸铁管及管件、钢铁冶炼"), _spec("钢铁")) == "CHAIN_STEEL_MID_PIPE"
|
||||
assert classify_chain_profile(_profile("水泥窑余热锅炉和节能环保设备"), _spec("建筑材料")) is None
|
||||
assert classify_chain_profile(_profile("贵金属和有色金属矿采选及金属贸易", product_name="合质金、铅锌精矿、黄金贸易"), _spec("有色金属")) == "CHAIN_METAL_PRECIOUS_GOLD"
|
||||
|
||||
|
||||
def test_latest_report_chains_reject_weak_concept_and_downstream_mentions():
|
||||
assert classify_chain_profile(None, _spec("房地产"), "公司参股房地产开发企业") is None
|
||||
assert classify_chain_profile(None, _spec("建筑材料"), "产品用于水泥熟料生产线") is None
|
||||
assert classify_chain_profile(None, _spec("钢铁"), "设备适用于热轧板卷生产") is None
|
||||
assert classify_chain_profile(None, _spec("有色金属"), "公司规划建设碳酸锂生产项目") is None
|
||||
assert classify_chain_profile(None, _spec("金融服务"), "为商业银行提供核心系统软件") is None
|
||||
assert classify_chain_profile(_profile("钢铁产品和金属软磁材料生产"), _spec("有色金属")) is None
|
||||
|
||||
|
||||
def test_coal_machinery_components_communications_and_aftermarket_report_leaves():
|
||||
assert classify_chain_profile(_profile("动力煤开采、洗选和动力煤生产"), _spec("煤化工")) == "CHAIN_COAL_UP_THERMAL"
|
||||
assert classify_chain_profile(_profile("炼焦煤生产，主要产品为主焦煤和配焦煤"), _spec("煤化工")) == "CHAIN_COAL_UP_COKING"
|
||||
assert classify_chain_profile(_profile("挖掘机整机、装载机整机研发制造"), _spec("基础设施工程")) == "CHAIN_INFRA_UP_EQ_EARTH"
|
||||
assert classify_chain_profile(_profile("高空作业平台和升降工作平台制造"), _spec("基础设施工程")) == "CHAIN_INFRA_UP_EQ_AERIAL"
|
||||
assert classify_chain_profile(_profile("柔性印制电路板和软硬结合板生产"), _spec("电子元件")) == "CHAIN_EC_PCB_FLEX"
|
||||
assert classify_chain_profile(_profile("铝电解电容器和薄膜电容器生产"), _spec("电子元件")) == "CHAIN_EC_PASSIVE_CAP"
|
||||
assert classify_chain_profile(_profile("移动通信基站和无线接入网设备"), _spec("通信网络服务")) == "CHAIN_COMM_MID_BASE"
|
||||
assert classify_chain_profile(_profile("基础电信业务和移动通信服务"), _spec("通信网络服务")) == "CHAIN_COMM_DOWN_OPERATOR"
|
||||
assert classify_chain_profile(_profile("二手车交易、经销和拍卖服务"), _spec("汽车后市场")) == "CHAIN_AF_DOWN_USED"
|
||||
assert classify_chain_profile(_profile("汽车维修保养和汽车美容服务"), _spec("汽车后市场")) == "CHAIN_AF_DOWN_REPAIR"
|
||||
|
||||
|
||||
def test_new_resource_and_service_chains_require_f10_business_evidence():
|
||||
assert classify_chain_profile(None, _spec("煤化工"), "公司设备用于动力煤开采") is None
|
||||
assert classify_chain_profile(None, _spec("基础设施工程"), "公司参股工程机械租赁企业") is None
|
||||
assert classify_chain_profile(None, _spec("电子元件"), "PCB概念、被动元件概念") is None
|
||||
assert classify_chain_profile(None, _spec("通信网络服务"), "通信设备概念") is None
|
||||
assert classify_chain_profile(None, _spec("汽车后市场"), "二手车概念") is None
|
||||
assert classify_chain_profile(_profile("汽车零部件用于维修替换市场"), _spec("汽车后市场")) is None
|
||||
|
||||
|
||||
def test_education_and_hr_keep_service_roles_separate():
|
||||
assert classify_chain_profile(_profile("在线招聘平台和招聘网站运营"), _spec("教育")) is None
|
||||
assert classify_chain_profile(_profile("在线招聘平台和招聘网站运营"), _spec("人力资源服务")) == "CHAIN_HR_UP_RECRUIT"
|
||||
assert classify_chain_profile(_profile("职业教育培训"), _spec("人力资源服务")) is None
|
||||
assert classify_chain_profile(_profile("职业教育培训"), _spec("教育")) == "CHAIN_EDU_MID_VOCATIONAL"
|
||||
assert classify_chain_profile(_profile("电机产品应用于扫地机器人"), _spec("家电")) is None
|
||||
assert classify_chain_profile(_profile("纺织助剂用于印染加工"), _spec("纺织服装")) is None
|
||||
assert classify_chain_profile(_profile("为银行提供业务流程外包"), _spec("人力资源服务")) is None
|
||||
assert classify_chain_profile(_profile("瓦楞纸箱设备和纸箱印刷机械"), _spec("轻工家居")) == "CHAIN_LIGHT_UP_EQUIPMENT"
|
||||
|
||||
|
||||
def test_business_context_rejects_downstream_incidental_and_planned_mentions():
|
||||
assert classify_chain_profile(None, _spec("家电"), "公司推出适用于家用服务机器人的3D视觉方案。") is None
|
||||
assert classify_chain_profile(None, _spec("轻工家居"), "软泡聚醚广泛用于软体家具产品的生产。") is None
|
||||
assert classify_chain_profile(None, _spec("教育"), "公司设有职业技能培训学校。") is None
|
||||
assert classify_chain_profile(None, _spec("教育"), "公司拟面向AI教育客户开展业务，当前收入规模较小。") is None
|
||||
assert classify_chain_profile(None, _spec("教育"), "公司捐资助学，帮助学校提升教育信息化水平。") is None
|
||||
assert classify_chain_profile(None, _spec("教育"), "公司与职业院校合作推动数字化人才培养。") is None
|
||||
assert classify_chain_profile(None, _spec("家电"), "主营小家电用温度控制器和小家电配线组件。") is None
|
||||
assert classify_chain_profile(
|
||||
None,
|
||||
_spec("教育"),
|
||||
"【F10主营介绍】主营业务：高精尖数字化人才培养；产品类型：短期培训；产品名称：AI开发课程；经营范围：教育咨询",
|
||||
) == "CHAIN_EDU_MID_VOCATIONAL"
|
||||
|
||||
|
||||
def test_equipment_context_wins_over_material_or_finished_product():
|
||||
assert classify_chain_profile(None, _spec("轻工家居"), "主营生活用纸智能装备和全自动纸巾生产线。") == "CHAIN_LIGHT_UP_EQUIPMENT"
|
||||
assert classify_chain_profile(None, _spec("轻工家居"), "主营人造板生产线高端装备。") == "CHAIN_LIGHT_UP_EQUIPMENT"
|
||||
|
||||
|
||||
def test_ths_operate_profile_parser_extracts_strong_business_fields():
|
||||
html = """
|
||||
<div id="intro"><h3>主营介绍</h3><ul>
|
||||
<li>主营业务： 综合人力资源服务。</li>
|
||||
<li>产品类型： 业务外包服务、薪酬福利服务、招聘及灵活用工服务</li>
|
||||
<li>产品名称： 人力资源SaaS</li>
|
||||
<li>经营范围： 人力资源服务；企业管理咨询。</li>
|
||||
</ul></div>
|
||||
"""
|
||||
assert parse_ths_operate_profile_html(html) == {
|
||||
"主营业务": "综合人力资源服务。",
|
||||
"产品类型": "业务外包服务、薪酬福利服务、招聘及灵活用工服务",
|
||||
"产品名称": "人力资源SaaS",
|
||||
"经营范围": "人力资源服务；企业管理咨询。",
|
||||
}
|
||||
assert classify_chain_profile(
|
||||
None,
|
||||
_spec("人力资源服务"),
|
||||
"【F10主营介绍】主营业务：计量检测服务；产品类型：检测服务；产品名称：计量仪器；经营范围：劳务派遣服务",
|
||||
) is None
|
||||
|
||||
|
||||
def test_operate_profile_parser_accepts_missing_optional_fields():
|
||||
assert parse_ths_operate_profile_html(
|
||||
'<div id="intro">主营业务：刻蚀设备研发生产</div>'
|
||||
) == {"主营业务": "刻蚀设备研发生产"}
|
||||
assert parse_ths_operate_profile_html(
|
||||
'<div id="intro">主营业务:刻蚀设备 产品名称:薄膜沉积设备</div>'
|
||||
) == {"主营业务": "刻蚀设备", "产品名称": "薄膜沉积设备"}
|
||||
|
||||
|
||||
@pytest.mark.parametrize("html", [
|
||||
"<html>访问验证，请稍后重试</html>",
|
||||
'<div id="intro">主营业务： 产品类型： 产品名称： 经营范围：投资管理</div>',
|
||||
])
|
||||
def test_operate_fetch_rejects_success_status_without_business_fields(monkeypatch, html):
|
||||
from types import SimpleNamespace
|
||||
import requests
|
||||
|
||||
response = SimpleNamespace(status_code=200, content=html.encode("utf-8"), raise_for_status=lambda: None)
|
||||
monkeypatch.setattr(requests, "get", lambda *_args, **_kwargs: response)
|
||||
monkeypatch.setattr(industry_chain, "_wait_for_f10_request_slot", lambda: None)
|
||||
with pytest.raises(ValueError, match="无有效业务字段"):
|
||||
industry_chain.fetch_ths_stock_operate_evidence("002371")
|
||||
|
||||
|
||||
def test_multi_board_specs_cover_distinct_candidate_segments():
|
||||
assert ("gn", "300756") in _spec("软件云安全").additional_boards
|
||||
assert ("thshy", "881109") in _spec("石化精细化工").additional_boards
|
||||
assert ("thshy", "881274") in _spec("传媒内容").additional_boards
|
||||
assert ("thshy", "881275") in _spec("传媒内容").additional_boards
|
||||
assert ("thshy", "881148") in _spec("交通物流").additional_boards
|
||||
assert ("thshy", "881146") in _spec("公用事业").additional_boards
|
||||
assert ("thshy", "881284") in _spec("环保").additional_boards
|
||||
|
||||
|
||||
def test_chain_role_terms_keep_suppliers_and_services_out_of_product_leaves():
|
||||
assert classify_chain_profile(_profile("eVTOL电池系统研发"), _spec("低空经济")) == "CHAIN_LA_UP_POWER"
|
||||
assert classify_chain_profile(_profile("细胞与基因治疗CDMO服务"), _spec("创新药")) == "CHAIN_DRUG_UP_CDMO"
|
||||
assert classify_chain_profile(_profile("医疗器械检测认证服务"), _spec("医疗器械")) == "CHAIN_MED_UP_CRO"
|
||||
|
||||
|
||||
def test_hangtiangongcheng_maps_to_coal_gasification_equipment():
|
||||
coal_chemical = CHAIN_SPECS[5]
|
||||
profile = _profile(
|
||||
"煤气化技术及关键设备的研发、工程设计、技术服务、设备成套供应及工程总承包。",
|
||||
"煤炭清洁高效利用业务、高端装备制造业务、工业气体运营业务",
|
||||
"煤炭清洁高效利用业务、高端装备制造业务、工业气体运营业务",
|
||||
"加工气化炉及关键设备；施工总承包；工程勘察设计。",
|
||||
)
|
||||
|
||||
assert classify_chain_profile(profile, coal_chemical) == "CHAIN_COAL_MID_GASIFICATION"
|
||||
|
||||
|
||||
def test_shanghai_gangwan_maps_to_geotechnical_foundation_engineering():
|
||||
infrastructure = CHAIN_SPECS[6]
|
||||
profile = _profile(
|
||||
"为全球客户提供涵盖勘察、设计、施工、监测的一站式岩土工程综合服务。",
|
||||
"地基处理、桩基工程、新能源基建",
|
||||
"地基处理、桩基工程、新能源基建",
|
||||
"软基础处理施工、地基与基础工程施工、航道工程施工、水利工程施工。",
|
||||
)
|
||||
|
||||
assert classify_chain_profile(profile, infrastructure) == "CHAIN_INFRA_MID_GEOTECH"
|
||||
|
||||
|
||||
def test_ths_concept_parser_extracts_concept_explanations():
|
||||
html = """
|
||||
<table>
|
||||
<thead><tr><th>概念名称</th><th>概念解析</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>商业航天</td><td>子公司提供卫星电源系统。展开</td></tr>
|
||||
<tr><td>其他</td><td>nan</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
"""
|
||||
|
||||
assert parse_ths_concept_evidence_html(html) == "子公司提供卫星电源系统。"
|
||||
|
||||
|
||||
def test_ths_concept_parser_deduplicates_repeated_expanded_sentences():
|
||||
html = """
|
||||
<table><thead><tr><th>概念名称</th><th>概念解析</th></tr></thead><tbody>
|
||||
<tr><td>宠物经济</td><td>公司生产宠物训练器。公司生产宠物训练器。展开</td></tr>
|
||||
</tbody></table>
|
||||
"""
|
||||
|
||||
assert parse_ths_concept_evidence_html(html) == "公司生产宠物训练器。"
|
||||
|
||||
|
||||
def test_ths_concept_parser_removes_capital_operation_history():
|
||||
html = """
|
||||
<table><thead><tr><th>概念名称</th><th>概念解析</th></tr></thead><tbody>
|
||||
<tr><td>旅游</td><td>公司从事豪华游轮运营业务，并于2015年启动重大资产重组。现主营网络游戏。展开</td></tr>
|
||||
</tbody></table>
|
||||
"""
|
||||
|
||||
assert parse_ths_concept_evidence_html(html) == "现主营网络游戏。"
|
||||
|
||||
|
||||
def test_ths_concept_name_alone_is_not_business_evidence():
|
||||
html = """
|
||||
<table>
|
||||
<thead><tr><th>概念名称</th><th>概念解析</th></tr></thead>
|
||||
<tbody><tr><td>细胞治疗</td><td>公司尚无相关产品收入。</td></tr></tbody>
|
||||
</table>
|
||||
"""
|
||||
|
||||
evidence = parse_ths_concept_evidence_html(html)
|
||||
assert "细胞治疗" not in evidence
|
||||
assert classify_chain_profile(None, _spec("创新药"), evidence) is None
|
||||
|
||||
|
||||
def test_ths_concept_parser_ignores_expanded_text_misread_as_a_name():
|
||||
long_name = "公司围绕主营业务持续开展技术研发和市场拓展并形成完整解决方案" * 3
|
||||
html = f"""
|
||||
<table>
|
||||
<thead><tr><th>概念名称</th><th>概念解析</th></tr></thead>
|
||||
<tbody>
|
||||
<tr><td>商业航天</td><td>提供卫星电源系统。</td></tr>
|
||||
<tr><td>{long_name}</td><td>展开后的详细说明。</td></tr>
|
||||
</tbody>
|
||||
</table>
|
||||
"""
|
||||
|
||||
assert industry_chain.parse_ths_concepts_html(html) == [
|
||||
{"name": "商业航天", "explanation": "提供卫星电源系统。"},
|
||||
]
|
||||
|
||||
|
||||
def test_shanghai_gangwan_concept_evidence_adds_space_power_membership():
|
||||
space = CHAIN_SPECS[2]
|
||||
profile = _profile(
|
||||
"为全球客户提供涵盖勘察、设计、施工、监测的一站式岩土工程综合服务。",
|
||||
"地基处理、桩基工程、新能源基建",
|
||||
"地基处理、桩基工程、新能源基建",
|
||||
"软基础处理施工、地基与基础工程施工、航道工程施工、水利工程施工。",
|
||||
)
|
||||
evidence = (
|
||||
"控股子公司上海伏曦炘空科技有限公司提供轻量化、低成本空间能源系统解决方案，"
|
||||
"已有40余套卫星电源系统、太阳帆板及结构件在轨运行。"
|
||||
)
|
||||
|
||||
assert classify_chain_profile(profile, space) is None
|
||||
assert classify_chain_profile(profile, space, evidence) == "CHAIN_SPACE_UP_POWER"
|
||||
|
||||
|
||||
def test_sync_classifies_board_constituent_only_from_ths_f10(monkeypatch):
|
||||
space = CHAIN_SPECS[2]
|
||||
written_rows = []
|
||||
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_: ["605598"])
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda *_: set())
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"fetch_ths_stock_chain_evidence",
|
||||
lambda *_: "子公司提供空间能源系统和卫星电源系统。",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"batch_upsert",
|
||||
lambda model, rows, _keys: written_rows.extend(rows) if model is industry_chain.StockIndustry else None,
|
||||
)
|
||||
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
space,
|
||||
symbol="605598",
|
||||
limit=None,
|
||||
refresh=True,
|
||||
workers=1,
|
||||
evidence_cache={},
|
||||
)
|
||||
|
||||
assert (matched, unmatched, failed) == (1, 0, 0)
|
||||
assert written_rows[0]["industry_code"] == "CHAIN_SPACE_UP_POWER"
|
||||
|
||||
|
||||
def test_sync_reclassifies_existing_constituent_missing_from_accessible_board_pages(monkeypatch):
|
||||
infrastructure = CHAIN_SPECS[6]
|
||||
written_rows = []
|
||||
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_: [])
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda *_: {"605598"})
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"fetch_ths_stock_chain_evidence",
|
||||
lambda *_: "【F10主营介绍】主营业务：地基处理、桩基工程和岩土工程综合服务；产品类型：岩土工程；产品名称：桩基工程；经营范围：工程施工",
|
||||
)
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"batch_upsert",
|
||||
lambda model, rows, _keys: written_rows.extend(rows) if model is industry_chain.StockIndustry else None,
|
||||
)
|
||||
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
infrastructure,
|
||||
symbol="605598",
|
||||
limit=None,
|
||||
refresh=True,
|
||||
workers=1,
|
||||
evidence_cache={},
|
||||
)
|
||||
|
||||
assert (matched, unmatched, failed) == (1, 0, 0)
|
||||
assert written_rows[0]["industry_code"] == "CHAIN_INFRA_MID_GEOTECH"
|
||||
|
||||
|
||||
def test_sync_discovers_cached_f10_match_outside_accessible_board_pages(monkeypatch):
|
||||
pet = _spec("宠物")
|
||||
written_rows = []
|
||||
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_: [])
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda *_: set())
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"batch_upsert",
|
||||
lambda model, rows, _keys: written_rows.extend(rows) if model is industry_chain.StockIndustry else None,
|
||||
)
|
||||
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
pet,
|
||||
symbol=None,
|
||||
limit=None,
|
||||
refresh=False,
|
||||
workers=1,
|
||||
evidence_cache={"300673": "主营宠物食品的研发、制造和销售。"},
|
||||
)
|
||||
|
||||
assert (matched, unmatched, failed) == (1, 0, 0)
|
||||
assert written_rows[0]["stock_code"] == "300673"
|
||||
assert written_rows[0]["industry_code"] == "CHAIN_PET_MID_FOOD"
|
||||
|
||||
|
||||
def test_sync_uses_cached_f10_matches_when_candidate_board_is_unavailable(monkeypatch):
|
||||
components = _spec("电子元件")
|
||||
written_rows = []
|
||||
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_: (_ for _ in ()).throw(RuntimeError("board unavailable")))
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda *_: set())
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"batch_upsert",
|
||||
lambda model, rows, _keys: written_rows.extend(rows) if model is industry_chain.StockIndustry else None,
|
||||
)
|
||||
|
||||
evidence = "【F10主营介绍】主营业务：柔性印制电路板生产；产品类型：柔性线路板；产品名称：FPC产品；经营范围：电子元件制造"
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
components,
|
||||
symbol=None,
|
||||
limit=None,
|
||||
refresh=False,
|
||||
workers=1,
|
||||
evidence_cache={"600001": evidence},
|
||||
)
|
||||
|
||||
assert (matched, unmatched, failed) == (1, 0, 0)
|
||||
assert written_rows[0]["industry_code"] == "CHAIN_EC_PCB_FLEX"
|
||||
|
||||
|
||||
def test_sync_preserves_unsupported_beijing_f10_code(monkeypatch):
|
||||
robot = CHAIN_SPECS[1]
|
||||
written_rows = []
|
||||
|
||||
monkeypatch.setattr(industry_chain, "fetch_ths_board_constituents", lambda *_: [])
|
||||
monkeypatch.setattr(industry_chain, "_existing_chain_codes", lambda *_: {"920415"})
|
||||
monkeypatch.setattr(
|
||||
industry_chain,
|
||||
"fetch_ths_stock_chain_evidence",
|
||||
lambda *_: (_ for _ in ()).throw(AssertionError("unsupported code should not be fetched")),
|
||||
)
|
||||
monkeypatch.setattr(industry_chain, "batch_upsert", lambda _model, rows, _keys: written_rows.extend(rows))
|
||||
|
||||
matched, unmatched, failed = industry_chain._sync_chain(
|
||||
robot,
|
||||
symbol="920415",
|
||||
limit=None,
|
||||
refresh=True,
|
||||
workers=1,
|
||||
evidence_cache={},
|
||||
)
|
||||
|
||||
assert (matched, unmatched, failed) == (0, 0, 1)
|
||||
assert written_rows == []
|
||||
|
||||
|
||||
def test_limit_up_thresholds_cover_st_growth_star_and_beijing_markets():
|
||||
assert industry_chain._is_limit_up_stock("600000", "普通股票", 9.9)
|
||||
assert industry_chain._is_limit_up_stock("600001", "ST测试", 4.9)
|
||||
assert industry_chain._is_limit_up_stock("300001", "创业板", 19.8)
|
||||
assert industry_chain._is_limit_up_stock("688001", "科创板", 19.8)
|
||||
assert industry_chain._is_limit_up_stock("920001", "北交所", 29.8)
|
||||
assert not industry_chain._is_limit_up_stock("300001", "创业板", 10.0)
|
||||
|
||||
|
||||
def test_dynamic_f10_tree_filters_market_tags_and_classifies_every_limit_up(monkeypatch):
|
||||
node_rows = []
|
||||
membership_rows = []
|
||||
deactivated = []
|
||||
concepts = {
|
||||
"605598": [
|
||||
{"name": "融资融券", "explanation": "交易标签"},
|
||||
{"name": "商业航天", "explanation": "卫星电源系统"},
|
||||
],
|
||||
"000001": [],
|
||||
}
|
||||
|
||||
monkeypatch.setattr(industry_chain, "_latest_limit_up_codes", lambda **_kwargs: (datetime(2026, 7, 10).date(), ["000001", "605598"]))
|
||||
monkeypatch.setattr(industry_chain, "_load_f10_concept_cache", lambda: concepts)
|
||||
monkeypatch.setattr(industry_chain, "_deactivate_existing_industry_classifications", lambda *_: deactivated.append(True))
|
||||
|
||||
def capture(model, rows, _keys):
|
||||
if model is industry_chain.IndustryNode:
|
||||
node_rows.extend(rows)
|
||||
elif model is industry_chain.StockIndustry:
|
||||
membership_rows.extend(rows)
|
||||
|
||||
monkeypatch.setattr(industry_chain, "batch_upsert", capture)
|
||||
|
||||
node_count, membership_count, failed = industry_chain._sync_limit_up_f10_tree(
|
||||
symbol=None,
|
||||
limit=None,
|
||||
refresh=False,
|
||||
workers=1,
|
||||
)
|
||||
|
||||
assert (node_count, membership_count, failed) == (13, 2, 0)
|
||||
names = {row["industry_name"] for row in node_rows}
|
||||
assert {"同花顺 F10 产业链", "半导体产业链", "商业航天", "F10暂无分类"}.issubset(names)
|
||||
assert {"原材料", "设备", "EDA/IP", "设计", "制造", "封测", "IDM/分立器件", "下游应用", "综合"}.issubset(names)
|
||||
assert all(
|
||||
row["parent_code"] == "F10_CHAIN_ROOT"
|
||||
for row in node_rows if row["industry_level"] == 2
|
||||
)
|
||||
assert {row["stock_code"] for row in membership_rows} == {"000001", "605598"}
|
||||
assert all(row["source"].startswith("industry_chain_") for row in membership_rows)
|
||||
assert deactivated == [True]
|
||||
|
||||
|
||||
def test_semiconductor_f10_evidence_maps_to_chain_stages_and_removes_flat_concepts():
|
||||
concepts = [
|
||||
{"name": "芯片概念", "explanation": "公司提供半导体设备和集成电路封测服务。"},
|
||||
{"name": "融资融券", "explanation": "交易标签"},
|
||||
]
|
||||
|
||||
assert industry_chain._semiconductor_stage_names(concepts) == ["设备", "封测"]
|
||||
assert industry_chain._industry_concept_names(concepts) == []
|
||||
|
||||
|
||||
def test_semiconductor_generic_f10_concept_falls_back_to_comprehensive_stage():
|
||||
concepts = [{"name": "半导体概念", "explanation": "公司相关业务正在推进。"}]
|
||||
|
||||
assert industry_chain._semiconductor_stage_names(concepts) == ["综合"]
|
||||
@@ -0,0 +1,149 @@
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
import pytest
|
||||
from sqlalchemy import create_engine, event, insert, select
|
||||
from sqlalchemy.exc import IntegrityError, StatementError
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
from src.db import StockIndustry
|
||||
from src.db import industry
|
||||
|
||||
|
||||
NOW = datetime(2026, 9, 25, 12)
|
||||
|
||||
|
||||
def membership(code="002371", node="ETCH", source="industry_chain", **values):
|
||||
return {"source": source, "stock_code": code, "industry_code": node,
|
||||
"updated_at": NOW, "effective_date": None, **values}
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def database(monkeypatch):
|
||||
engine = create_engine("sqlite://")
|
||||
StockIndustry.__table__.create(engine)
|
||||
locks = []
|
||||
|
||||
@event.listens_for(engine, "before_cursor_execute", retval=True)
|
||||
def advisory_lock_stub(_conn, _cursor, statement, params, _context, _many):
|
||||
# Only MySQL's connection-lock calls are stubbed; all membership SQL runs.
|
||||
if "GET_LOCK" in statement or "RELEASE_LOCK" in statement:
|
||||
locks.append(statement)
|
||||
return "SELECT 1", ()
|
||||
return statement, params
|
||||
|
||||
monkeypatch.setattr(industry, "get_engine", lambda: engine)
|
||||
yield engine, locks
|
||||
engine.dispose()
|
||||
|
||||
|
||||
def stored(engine, *, include_inactive=False):
|
||||
with Session(engine) as session:
|
||||
query = select(StockIndustry.id, StockIndustry.source, StockIndustry.stock_code,
|
||||
StockIndustry.industry_code, StockIndustry.updated_at, StockIndustry.active)
|
||||
if not include_inactive:
|
||||
query = query.where(StockIndustry.active.is_(True))
|
||||
return {(source, code, node): (row_id, updated, active)
|
||||
for row_id, source, code, node, updated, active in session.execute(query)}
|
||||
|
||||
|
||||
def test_multiple_memberships_are_idempotent_and_preserve_other_stock_and_source(database):
|
||||
engine, _ = database
|
||||
untouched = [membership(code="688012"), membership(source="other")]
|
||||
industry.replace_stock_industry_memberships([membership(), *untouched])
|
||||
before = stored(engine)
|
||||
rows = [membership(updated_at=NOW + timedelta(days=1)), membership(node="DEPOSITION")]
|
||||
industry.replace_stock_industry_memberships([*rows, rows[1]])
|
||||
after = stored(engine)
|
||||
assert len(after) == 4
|
||||
assert after[("industry_chain", "002371", "ETCH")][0] == before[("industry_chain", "002371", "ETCH")][0]
|
||||
assert after[("industry_chain", "002371", "ETCH")][1] == NOW + timedelta(days=1)
|
||||
for row in untouched:
|
||||
key = row["source"], row["stock_code"], row["industry_code"]
|
||||
assert after[key] == before[key]
|
||||
industry.replace_stock_industry_memberships(rows)
|
||||
assert stored(engine) == after
|
||||
|
||||
|
||||
def test_refresh_replaces_stale_memberships_and_unmatched_placeholder(database):
|
||||
engine, _ = database
|
||||
industry.replace_stock_industry_memberships([membership(), membership(node="DEPOSITION")])
|
||||
original_ids = {key: value[0] for key, value in stored(engine).items()}
|
||||
industry.replace_stock_industry_memberships([membership(node="CLEAN")])
|
||||
assert set(stored(engine)) == {("industry_chain", "002371", "CLEAN")}
|
||||
industry.replace_stock_industry_memberships([membership(node="CHAIN_SEMI_UNMATCHED")])
|
||||
assert set(stored(engine)) == {("industry_chain", "002371", "CHAIN_SEMI_UNMATCHED")}
|
||||
industry.replace_stock_industry_memberships([membership(), membership(node="DEPOSITION")])
|
||||
assert {key[2] for key in stored(engine)} == {"ETCH", "DEPOSITION"}
|
||||
assert {key: value[0] for key, value in stored(engine).items()} == original_ids
|
||||
all_rows = stored(engine, include_inactive=True)
|
||||
assert len(all_rows) == 4
|
||||
assert all_rows[("industry_chain", "002371", "CLEAN")][2] is False
|
||||
assert all_rows[("industry_chain", "002371", "CHAIN_SEMI_UNMATCHED")][2] is False
|
||||
|
||||
|
||||
def test_failed_insert_rolls_back_updates_and_releases_lock(database):
|
||||
engine, locks = database
|
||||
industry.replace_stock_industry_memberships([membership()])
|
||||
before = stored(engine)
|
||||
with pytest.raises(StatementError):
|
||||
industry.replace_stock_industry_memberships([
|
||||
membership(updated_at=NOW + timedelta(days=1)),
|
||||
membership(node="DEPOSITION", effective_date="invalid-date"),
|
||||
])
|
||||
assert stored(engine) == before
|
||||
assert len(locks) == 4
|
||||
assert "GET_LOCK" in locks[-2] and "RELEASE_LOCK" in locks[-1]
|
||||
|
||||
|
||||
def test_batches_never_split_a_stocks_memberships(database):
|
||||
engine, locks = database
|
||||
rows = [membership(code=f"{code:06}", node=node) for code in range(101) for node in ("ETCH", "DEPOSITION")]
|
||||
industry.replace_stock_industry_memberships(rows)
|
||||
assert len(stored(engine)) == 202
|
||||
assert len(locks) == 2
|
||||
industry.replace_stock_industry_memberships(rows)
|
||||
assert len(stored(engine)) == 202
|
||||
|
||||
|
||||
def test_unique_constraint_allows_different_nodes_but_rejects_duplicate(database):
|
||||
engine, _ = database
|
||||
industry.replace_stock_industry_memberships([membership(), membership(node="DEPOSITION")])
|
||||
with Session(engine) as session, pytest.raises(IntegrityError):
|
||||
session.execute(insert(StockIndustry), [membership()])
|
||||
|
||||
|
||||
def test_empty_or_invalid_input_never_acquires_a_lock(database):
|
||||
_, locks = database
|
||||
industry.replace_stock_industry_memberships([])
|
||||
with pytest.raises(ValueError):
|
||||
industry.replace_stock_industry_memberships([membership(node="")])
|
||||
assert locks == []
|
||||
|
||||
|
||||
def test_retiring_source_retains_rows_and_other_sources(database):
|
||||
engine, _ = database
|
||||
industry.replace_stock_industry_memberships([membership(), membership(node="DEPOSITION"), membership(source="other")])
|
||||
before = stored(engine)
|
||||
industry.deactivate_stock_industry_sources(["industry_chain", "industry_chain"])
|
||||
assert set(stored(engine)) == {("other", "002371", "ETCH")}
|
||||
after = stored(engine, include_inactive=True)
|
||||
assert set(after) == set(before)
|
||||
assert all(after[key][0] == before[key][0] for key in before)
|
||||
|
||||
|
||||
def test_lock_timeout_does_not_write_or_release_another_owners_lock(database, monkeypatch):
|
||||
from sqlalchemy.engine import Connection
|
||||
|
||||
engine, locks = database
|
||||
original_scalar = Connection.scalar
|
||||
|
||||
def refuse_lock(connection, statement, *args, **kwargs):
|
||||
if "GET_LOCK" in str(statement):
|
||||
return 0
|
||||
return original_scalar(connection, statement, *args, **kwargs)
|
||||
|
||||
monkeypatch.setattr(Connection, "scalar", refuse_lock)
|
||||
with pytest.raises(RuntimeError, match="Could not acquire"):
|
||||
industry.replace_stock_industry_memberships([membership()])
|
||||
assert stored(engine, include_inactive=True) == {}
|
||||
assert locks == []
|
||||
@@ -0,0 +1,235 @@
|
||||
"""验证命令入口。"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src import main
|
||||
from src.app import commands as app_commands
|
||||
from src.app import main as app_main
|
||||
from src.fetchers import daily
|
||||
|
||||
|
||||
def test_current_packages_import():
|
||||
import src.app.bootstrap # noqa: F401
|
||||
import src.core.config # noqa: F401
|
||||
import src.core.log # noqa: F401
|
||||
import src.db.models # noqa: F401
|
||||
import src.db.runtime # noqa: F401
|
||||
import src.db.status # noqa: F401
|
||||
import src.sources.stock_list_source # noqa: F401
|
||||
import src.sources.trading_day_source # noqa: F401
|
||||
import src.services.daily_service # noqa: F401
|
||||
import src.services.minute_service # noqa: F401
|
||||
import src.services.reference_service # noqa: F401
|
||||
import src.services.trade_status_service # noqa: F401
|
||||
|
||||
|
||||
def test_daily_compat_entry_delegates_to_current_fetcher():
|
||||
with patch("src.fetchers.daily.fetch_tdx_daily") as mock_fetch:
|
||||
daily.fetch_daily(
|
||||
start_date="20260515",
|
||||
end_date="20260515",
|
||||
symbol="600498",
|
||||
)
|
||||
|
||||
mock_fetch.assert_called_once_with(
|
||||
start_date="20260515",
|
||||
end_date="20260515",
|
||||
symbol="600498",
|
||||
)
|
||||
|
||||
|
||||
def test_build_parser_selects_daily_command():
|
||||
parser = main.build_parser()
|
||||
|
||||
args = parser.parse_args(["--daily", "--symbol", "600498"])
|
||||
|
||||
assert main.selected_commands(args) == ["daily"]
|
||||
|
||||
|
||||
def test_app_main_re_exports_parser_helpers():
|
||||
assert app_main.build_parser is main.build_parser
|
||||
assert app_main.selected_commands is main.selected_commands
|
||||
assert app_main.run_command is main.run_command
|
||||
|
||||
|
||||
def test_main_entries_delegate_to_shared_bootstrap():
|
||||
from src.app import bootstrap
|
||||
|
||||
assert main.execute_cli is bootstrap.execute_cli
|
||||
assert app_main.execute_cli is bootstrap.execute_cli
|
||||
|
||||
|
||||
def test_app_commands_exposes_minute_dispatch_helpers():
|
||||
parser = app_commands.build_parser()
|
||||
args = parser.parse_args(["--minute"])
|
||||
|
||||
assert app_commands.selected_commands(args) == ["minute"]
|
||||
|
||||
|
||||
def test_daily_command_dispatches_current_fetcher():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.daily_service.sync_daily") as mock_sync, \
|
||||
patch("sys.argv", ["main.py", "--daily", "--start-date", "20260515", "--end-date", "20260515"]):
|
||||
main.main()
|
||||
|
||||
mock_sync.assert_called_once_with(
|
||||
start_date="20260515",
|
||||
end_date="20260515",
|
||||
symbol=None,
|
||||
limit=None,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
|
||||
def test_daily_command_forwards_symbol_and_limit():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.daily_service.sync_daily") as mock_sync, \
|
||||
patch("sys.argv", ["main.py", "--daily", "--symbol", "600498", "--start-date", "20260515", "--limit", "20", "--include-suspended"]):
|
||||
main.main()
|
||||
|
||||
mock_sync.assert_called_once_with(
|
||||
start_date="20260515",
|
||||
end_date=None,
|
||||
symbol="600498",
|
||||
limit=20,
|
||||
skip_suspended=False,
|
||||
)
|
||||
|
||||
|
||||
def test_backfill_command_dispatches_daily_and_minute_workflow():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.backfill_service.run_backfill") as mock_run, \
|
||||
patch("sys.argv", [
|
||||
"main.py",
|
||||
"--backfill",
|
||||
"--start-date", "20150101",
|
||||
"--limit", "10",
|
||||
]):
|
||||
main.main()
|
||||
|
||||
mock_run.assert_called_once_with(
|
||||
start_date="20150101",
|
||||
end_date=None,
|
||||
symbol=None,
|
||||
limit=10,
|
||||
daemon=False,
|
||||
interval_seconds=None,
|
||||
mode="auto",
|
||||
lookback_days=None,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
|
||||
def test_backfill_daemon_is_selected_without_explicit_backfill_flag():
|
||||
parser = main.build_parser()
|
||||
args = parser.parse_args(["--daemon", "--interval", "60"])
|
||||
|
||||
assert main.selected_commands(args) == ["backfill"]
|
||||
assert args.backfill_daemon is True
|
||||
assert args.backfill_interval == 60
|
||||
|
||||
|
||||
def test_stock_info_command_dispatches_bootstrap():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.reference_service.sync_stock_info") as mock_sync, \
|
||||
patch("sys.argv", ["main.py", "--stock-info"]):
|
||||
main.main()
|
||||
|
||||
mock_sync.assert_called_once_with()
|
||||
|
||||
|
||||
def test_trading_day_command_dispatches_current_fetcher():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.reference_service.sync_trading_days") as mock_sync, \
|
||||
patch("sys.argv", ["main.py", "--trading-day", "--start-date", "20260501", "--end-date", "20260517"]):
|
||||
main.main()
|
||||
|
||||
mock_sync.assert_called_once_with("20260501", "20260517")
|
||||
|
||||
|
||||
def test_minute_command_dispatches_fetcher():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.minute_service.sync_minute") as mock_sync, \
|
||||
patch("sys.argv", [
|
||||
"main.py",
|
||||
"--minute",
|
||||
"--symbol",
|
||||
"600519",
|
||||
"--start-date",
|
||||
"20260101",
|
||||
"--end-date",
|
||||
"20260618",
|
||||
"--limit",
|
||||
"200",
|
||||
]):
|
||||
main.main()
|
||||
|
||||
mock_sync.assert_called_once_with(
|
||||
symbol="600519",
|
||||
start_date="20260101",
|
||||
end_date="20260618",
|
||||
limit=200,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
|
||||
def test_trade_status_command_dispatches_fetcher():
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch("src.services.trade_status_service.archive_trade_status") as mock_archive, \
|
||||
patch("sys.argv", ["main.py", "--mark-suspended", "--start-date", "20260601", "--end-date", "20260618"]):
|
||||
main.main()
|
||||
|
||||
mock_archive.assert_called_once_with(start_date="20260601", end_date="20260618")
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"argv",
|
||||
[
|
||||
["main.py", "--stock-info"],
|
||||
["main.py", "--trading-day"],
|
||||
["main.py", "--daily"],
|
||||
["main.py", "--backfill"],
|
||||
["main.py", "--trade-status"],
|
||||
["main.py", "--minute"],
|
||||
],
|
||||
)
|
||||
def test_main_handles_keyboard_interrupt_for_all_commands(argv):
|
||||
with patch.object(main, "load_config"), \
|
||||
patch.object(main, "init_db"), \
|
||||
patch.object(main, "run_command", side_effect=KeyboardInterrupt), \
|
||||
patch.object(main._logger, "warning") as mock_warning, \
|
||||
patch("sys.argv", argv):
|
||||
main.main()
|
||||
|
||||
mock_warning.assert_called_once_with("任务被用户中断")
|
||||
|
||||
|
||||
def test_main_handles_keyboard_interrupt_during_startup():
|
||||
with patch.object(main, "load_config", side_effect=KeyboardInterrupt), \
|
||||
patch.object(main, "init_db") as mock_init_db, \
|
||||
patch.object(main._logger, "warning") as mock_warning, \
|
||||
patch("sys.argv", ["main.py", "--daily"]):
|
||||
main.main()
|
||||
|
||||
mock_init_db.assert_not_called()
|
||||
mock_warning.assert_called_once_with("任务被用户中断")
|
||||
|
||||
|
||||
def test_app_main_handles_keyboard_interrupt():
|
||||
with patch.object(app_main, "load_config"), \
|
||||
patch.object(app_main, "init_db"), \
|
||||
patch.object(app_main, "run_command", side_effect=KeyboardInterrupt), \
|
||||
patch.object(app_main._logger, "warning") as mock_warning, \
|
||||
patch("sys.argv", ["main.py", "--daily"]):
|
||||
app_main.main()
|
||||
|
||||
mock_warning.assert_called_once_with("任务被用户中断")
|
||||
@@ -1,28 +0,0 @@
|
||||
"""验证 market_daily._is_20pct 的板块判定"""
|
||||
|
||||
from src.fetchers.market_daily import _is_20pct
|
||||
|
||||
|
||||
def test_main_board_10pct():
|
||||
# 沪深主板 10% 限价
|
||||
assert _is_20pct("600000") is False
|
||||
assert _is_20pct("601318") is False
|
||||
assert _is_20pct("000001") is False
|
||||
assert _is_20pct("002594") is False
|
||||
|
||||
|
||||
def test_chinext_20pct():
|
||||
# 创业板 300 / 301 → 20%
|
||||
assert _is_20pct("300750") is True
|
||||
assert _is_20pct("301001") is True
|
||||
|
||||
|
||||
def test_kechuang_20pct():
|
||||
# 科创板 688 / 689 → 20%
|
||||
assert _is_20pct("688981") is True
|
||||
assert _is_20pct("689001") is True
|
||||
|
||||
|
||||
def test_beijing_not_20pct():
|
||||
# 北交所 920 系列即便涨跌幅 20%+，也不属于科创/创业 20% 板块计数
|
||||
assert _is_20pct("920001") is False
|
||||
@@ -0,0 +1,32 @@
|
||||
"""验证业务代码不包含删除式数据库操作。"""
|
||||
|
||||
from pathlib import Path
|
||||
import ast
|
||||
|
||||
|
||||
def test_no_delete_or_drop_statements_in_src():
|
||||
"""src 下不应包含 DELETE / DROP，ALTER 仅允许定向兼容补列。"""
|
||||
allowed_alter_files = set()
|
||||
for path in Path("src").rglob("*.py"):
|
||||
text = path.read_text(encoding="utf-8")
|
||||
tree = ast.parse(text, filename=str(path))
|
||||
for node in ast.walk(tree):
|
||||
if isinstance(node, ast.ImportFrom) and node.module and node.module.startswith("sqlalchemy"):
|
||||
assert all(item.name != "delete" for item in node.names), path
|
||||
if isinstance(node, ast.Call):
|
||||
name = node.func.id if isinstance(node.func, ast.Name) else node.func.attr if isinstance(node.func, ast.Attribute) else ""
|
||||
assert name not in {"delete", "drop_all"}, path
|
||||
assert "DELETE FROM" not in text, path
|
||||
assert "DROP TABLE" not in text, path
|
||||
assert "DROP TABLE IF EXISTS" not in text, path
|
||||
if path not in allowed_alter_files:
|
||||
assert "ALTER TABLE" not in text, path
|
||||
assert "ADD COLUMN" not in text, path
|
||||
|
||||
|
||||
def test_db_init_only_creates_tables():
|
||||
"""数据库初始化逻辑只应负责建表。"""
|
||||
text = Path("src/db/runtime.py").read_text(encoding="utf-8")
|
||||
assert "create_all" in text
|
||||
assert "ALTER TABLE" not in text
|
||||
assert "ADD COLUMN" not in text
|
||||
@@ -1,29 +0,0 @@
|
||||
"""验证 sector.fetch_sector 概念板块抓取逻辑"""
|
||||
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from src.fetchers import sector
|
||||
|
||||
|
||||
def test_concept_skips_when_enough_data():
|
||||
"""已有数量达到当前板块数量时，跳过抓取。"""
|
||||
mock_session = MagicMock()
|
||||
mock_session.execute.side_effect = [MagicMock(scalar=MagicMock(return_value=150000)),
|
||||
MagicMock(scalar=MagicMock(return_value=460))]
|
||||
with patch.object(sector, "get_session", return_value=mock_session), \
|
||||
patch.object(sector, "_fetch_concept_list", return_value=[{"code": "880001", "name": "测试板块"}]) as mock_list:
|
||||
sector.fetch_sector()
|
||||
|
||||
mock_list.assert_called_once_with()
|
||||
|
||||
|
||||
def test_concept_fetches_when_insufficient():
|
||||
"""板块数量不足时，执行抓取。"""
|
||||
mock_session = MagicMock()
|
||||
mock_session.execute.side_effect = [MagicMock(scalar=MagicMock(return_value=0)),
|
||||
MagicMock(scalar=MagicMock(return_value=0))]
|
||||
|
||||
with patch.object(sector, "get_session", return_value=mock_session), \
|
||||
patch.object(sector, "_fetch_concept_list", return_value=[{"code": "880001", "name": "测试板块"}]), \
|
||||
patch.object(sector, "_fetch_concept_stocks", return_value=[]):
|
||||
sector.fetch_sector()
|
||||
@@ -0,0 +1,151 @@
|
||||
"""验证服务层编排入口。"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.services import backfill_service, daily_service, minute_service, reference_service, trade_status_service
|
||||
|
||||
|
||||
def test_daily_service_forwards_arguments():
|
||||
with patch.object(daily_service, "fetch_tdx_daily") as mock_fetch:
|
||||
daily_service.sync_daily(
|
||||
start_date="20260501",
|
||||
end_date="20260517",
|
||||
symbol="600498",
|
||||
limit=20,
|
||||
)
|
||||
|
||||
mock_fetch.assert_called_once_with(
|
||||
start_date="20260501",
|
||||
end_date="20260517",
|
||||
symbol="600498",
|
||||
limit=20,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
|
||||
def test_minute_service_forwards_arguments():
|
||||
with patch.object(minute_service, "fetch_tdx_minute") as mock_fetch:
|
||||
minute_service.sync_minute(
|
||||
symbol="600519",
|
||||
start_date="20260101",
|
||||
end_date="20260618",
|
||||
limit=100,
|
||||
)
|
||||
|
||||
mock_fetch.assert_called_once_with(
|
||||
symbol="600519",
|
||||
start_date="20260101",
|
||||
end_date="20260618",
|
||||
limit=100,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
|
||||
def test_backfill_runs_daily_minute_then_index_daily():
|
||||
with patch.object(backfill_service, "sync_daily") as mock_daily, \
|
||||
patch.object(backfill_service, "sync_minute") as mock_minute, \
|
||||
patch.object(backfill_service, "sync_index_daily") as mock_index:
|
||||
backfill_service.backfill_once(
|
||||
start_date="20260501",
|
||||
end_date="20260517",
|
||||
symbol="600519",
|
||||
limit=20,
|
||||
skip_suspended=True,
|
||||
)
|
||||
|
||||
mock_daily.assert_called_once_with(
|
||||
start_date="20260501",
|
||||
end_date="20260517",
|
||||
symbol="600519",
|
||||
limit=20,
|
||||
skip_suspended=True,
|
||||
)
|
||||
mock_minute.assert_called_once_with(
|
||||
symbol="600519",
|
||||
start_date="20260501",
|
||||
end_date="20260517",
|
||||
limit=20,
|
||||
skip_suspended=True,
|
||||
)
|
||||
mock_index.assert_called_once_with(start_date="20260501", end_date="20260517")
|
||||
|
||||
|
||||
def test_backfill_continues_to_minute_when_daily_fails():
|
||||
with patch.object(backfill_service, "sync_daily", side_effect=RuntimeError("daily")), \
|
||||
patch.object(backfill_service, "sync_minute") as mock_minute, \
|
||||
patch.object(backfill_service, "sync_index_daily") as mock_index:
|
||||
with pytest.raises(RuntimeError, match="日K"):
|
||||
backfill_service.backfill_once(start_date="20260501", end_date="20260517")
|
||||
|
||||
mock_minute.assert_called_once()
|
||||
mock_index.assert_called_once()
|
||||
|
||||
|
||||
def test_backfill_reports_index_daily_failure_after_other_datasets_run():
|
||||
with patch.object(backfill_service, "sync_daily") as mock_daily, \
|
||||
patch.object(backfill_service, "sync_minute") as mock_minute, \
|
||||
patch.object(backfill_service, "sync_index_daily", side_effect=RuntimeError("index")):
|
||||
with pytest.raises(RuntimeError, match="指数日线"):
|
||||
backfill_service.backfill_once(start_date="20260501", end_date="20260517")
|
||||
|
||||
mock_daily.assert_called_once()
|
||||
mock_minute.assert_called_once()
|
||||
|
||||
|
||||
def test_backfill_daemon_retries_after_cycle_failure():
|
||||
calls = []
|
||||
|
||||
def fake_once(**kwargs):
|
||||
calls.append(kwargs)
|
||||
if len(calls) == 1:
|
||||
raise RuntimeError("temporary")
|
||||
|
||||
def stop_after_sleep(_seconds):
|
||||
raise KeyboardInterrupt
|
||||
|
||||
with patch.object(backfill_service, "backfill_once", side_effect=fake_once), \
|
||||
patch.object(backfill_service.time, "sleep", side_effect=stop_after_sleep):
|
||||
with pytest.raises(KeyboardInterrupt):
|
||||
backfill_service.run_backfill(daemon=True, interval_seconds=1)
|
||||
|
||||
assert len(calls) == 1
|
||||
|
||||
|
||||
def test_incremental_start_date_uses_lookback_and_configured_lower_bound():
|
||||
assert backfill_service._incremental_start_date(
|
||||
backfill_service.date(2026, 7, 10),
|
||||
end_date="20260711",
|
||||
configured_start="20150101",
|
||||
lookback_days=30,
|
||||
) == "20260610"
|
||||
assert backfill_service._incremental_start_date(
|
||||
backfill_service.date(2015, 1, 10),
|
||||
end_date="20260711",
|
||||
configured_start="20150101",
|
||||
lookback_days=30,
|
||||
) == "20150101"
|
||||
|
||||
|
||||
def test_reference_service_forwards_stock_info():
|
||||
with patch.object(reference_service, "bootstrap_stock_list", return_value=["000001"]) as mock_fetch:
|
||||
rows = reference_service.sync_stock_info()
|
||||
|
||||
assert rows == ["000001"]
|
||||
mock_fetch.assert_called_once_with()
|
||||
|
||||
|
||||
def test_reference_service_forwards_trading_days():
|
||||
with patch.object(reference_service, "fetch_trading_days", return_value=["2026-05-08"]) as mock_fetch:
|
||||
rows = reference_service.sync_trading_days("20260501", "20260517")
|
||||
|
||||
assert rows == ["2026-05-08"]
|
||||
mock_fetch.assert_called_once_with("20260501", "20260517")
|
||||
|
||||
|
||||
def test_trade_status_service_forwards_arguments():
|
||||
with patch.object(trade_status_service, "fetch_trade_status") as mock_fetch:
|
||||
trade_status_service.archive_trade_status(start_date="20260601", end_date="20260618")
|
||||
|
||||
mock_fetch.assert_called_once_with(start_date="20260601", end_date="20260618")
|
||||
@@ -0,0 +1,20 @@
|
||||
"""验证股票列表兼容入口。"""
|
||||
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.fetchers import stock_list
|
||||
|
||||
|
||||
def test_fetch_stock_list_delegates_to_bootstrap():
|
||||
with patch.object(stock_list, "bootstrap_stock_list", return_value=["000001", "600000"]) as mock_bootstrap:
|
||||
rows = stock_list.fetch_stock_list()
|
||||
|
||||
assert rows == ["000001", "600000"]
|
||||
mock_bootstrap.assert_called_once_with()
|
||||
|
||||
|
||||
def test_fetch_stock_list_rows_is_export():
|
||||
with patch.object(stock_list, "fetch_stock_list_rows", return_value=[{"code": "000001", "name": "平安银行"}]):
|
||||
rows = stock_list.fetch_stock_list_rows()
|
||||
|
||||
assert rows == [{"code": "000001", "name": "平安银行"}]
|
||||
@@ -0,0 +1,64 @@
|
||||
"""验证停牌标记的查询和抓取跳过行为。"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
from datetime import date
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.db import status as db_status
|
||||
from src.fetchers import tdx_daily, tdx_minute
|
||||
|
||||
|
||||
def test_suspended_marker_query_groups_days_by_code():
|
||||
fake_session = SimpleNamespace(
|
||||
execute=lambda _statement: [
|
||||
("600000", date(2026, 5, 16)),
|
||||
("600000", date(2026, 5, 17)),
|
||||
("000001", date(2026, 5, 17)),
|
||||
],
|
||||
close=lambda: None,
|
||||
)
|
||||
|
||||
with patch.object(db_status, "get_session", return_value=fake_session):
|
||||
result = db_status.get_suspended_trade_days(["600000", "000001"], "20260516", "20260517")
|
||||
|
||||
assert result == {
|
||||
"600000": {"2026-05-16", "2026-05-17"},
|
||||
"000001": {"2026-05-17"},
|
||||
}
|
||||
|
||||
|
||||
def test_daily_skips_days_marked_suspended():
|
||||
with patch.object(tdx_daily, "get_fetch_config", return_value={"delay": 0}), \
|
||||
patch.object(tdx_daily, "get_stock_codes", return_value=["600000"]), \
|
||||
patch.object(tdx_daily, "fetch_trading_days", return_value=["2026-05-16", "2026-05-17"]), \
|
||||
patch.object(tdx_daily, "get_ipo_dates", return_value={}), \
|
||||
patch.object(tdx_daily, "_load_existing_daily_dates", return_value={}), \
|
||||
patch.object(tdx_daily, "get_suspended_trade_days", return_value={"600000": {"2026-05-16"}}), \
|
||||
patch.object(tdx_daily, "_fetch_one_stock_with_api_retry", return_value=[]) as mock_fetch, \
|
||||
patch.object(tdx_daily, "connected_session") as mock_connected, \
|
||||
patch.object(tdx_daily.time, "sleep", return_value=None):
|
||||
mock_connected.return_value.__enter__.return_value = object()
|
||||
tdx_daily.fetch_tdx_daily(start_date="20260516", end_date="20260517")
|
||||
|
||||
assert mock_fetch.call_args.args[2:4] == ("20260517", "20260517")
|
||||
|
||||
|
||||
def test_minute_skips_symbols_with_only_suspended_missing_days():
|
||||
@contextmanager
|
||||
def forbidden_connection():
|
||||
raise AssertionError("停牌标记应在连接行情服务器前跳过")
|
||||
yield
|
||||
|
||||
with patch.object(tdx_minute, "get_fetch_config", return_value={"delay": 0}), \
|
||||
patch.object(tdx_minute, "_load_trading_days", return_value=["2026-05-17"]), \
|
||||
patch.object(tdx_minute, "_get_existing_minute_days", return_value={}), \
|
||||
patch.object(tdx_minute, "_get_latest_minute_dates", return_value={}), \
|
||||
patch.object(tdx_minute, "get_suspended_trade_days", return_value={"600519": {"2026-05-17"}}), \
|
||||
patch.object(tdx_minute, "fetch_market_indices_min5"), \
|
||||
patch.object(tdx_minute.tdx_daily, "connected_session", forbidden_connection):
|
||||
tdx_minute.fetch_tdx_minute(
|
||||
symbol="600519",
|
||||
start_date="20260517",
|
||||
end_date="20260517",
|
||||
)
|
||||
@@ -0,0 +1,151 @@
|
||||
"""验证日线数据转换逻辑。"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
|
||||
from src.fetchers import tdx_daily
|
||||
|
||||
|
||||
def test_bars_to_rows_converts_and_fills_derived_fields():
|
||||
bars = [
|
||||
{
|
||||
"datetime": "2026-05-08",
|
||||
"open": 10.0,
|
||||
"high": 10.5,
|
||||
"low": 9.9,
|
||||
"close": 10.0,
|
||||
"vol": 100,
|
||||
"amount": 1000,
|
||||
},
|
||||
{
|
||||
"datetime": "2026-05-11",
|
||||
"open": 10.2,
|
||||
"high": 10.8,
|
||||
"low": 10.1,
|
||||
"close": 10.5,
|
||||
"vol": 120,
|
||||
"amount": 1260,
|
||||
},
|
||||
]
|
||||
|
||||
rows = tdx_daily._tdx_bars_to_rows("sh.600000", bars, start_date="20260508", end_date="20260511")
|
||||
|
||||
assert len(rows) == 1
|
||||
assert rows[0]["code"] == "600000"
|
||||
assert rows[0]["date"] == "2026-05-11"
|
||||
assert rows[0]["pct_change"] == 5.0
|
||||
assert rows[0]["change"] == 0.5
|
||||
assert rows[0]["amplitude"] == 7.0
|
||||
assert rows[0]["turnover_rate"] == 0.0
|
||||
|
||||
|
||||
def test_infer_market_for_sh_sz_codes():
|
||||
assert tdx_daily.infer_tdx_market("600000") == 1
|
||||
assert tdx_daily.infer_tdx_market("000001") == 0
|
||||
assert tdx_daily.infer_tdx_market("300001") == 0
|
||||
try:
|
||||
tdx_daily.infer_tdx_market("399001")
|
||||
except ValueError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("expected index code 399001 to be skipped")
|
||||
|
||||
|
||||
def test_fetch_stock_list_rows_pages_sparse_security_list(monkeypatch):
|
||||
class FakeApi:
|
||||
def get_security_count(self, market):
|
||||
return 2000
|
||||
|
||||
def get_security_list(self, market, start):
|
||||
if market == 0 and start == 0:
|
||||
return [{"code": "000001", "name": "平安银行"}]
|
||||
if market == 0 and start == 1000:
|
||||
return []
|
||||
if market == 1 and start == 0:
|
||||
return []
|
||||
if market == 1 and start == 1000:
|
||||
return [{"code": "600000", "name": "浦发银行"}]
|
||||
return []
|
||||
|
||||
@contextmanager
|
||||
def fake_connected_session():
|
||||
yield FakeApi()
|
||||
|
||||
monkeypatch.setattr(tdx_daily, "connected_session", fake_connected_session)
|
||||
|
||||
rows = tdx_daily.fetch_stock_list_rows()
|
||||
|
||||
assert rows == [
|
||||
{"code": "000001", "name": "平安银行", "ipo_date": None},
|
||||
{"code": "600000", "name": "浦发银行", "ipo_date": None},
|
||||
]
|
||||
|
||||
|
||||
def test_fetch_stock_list_rows_skips_bad_security_page(monkeypatch):
|
||||
class FakeApi:
|
||||
def get_security_count(self, market):
|
||||
return 2000 if market == 0 else 0
|
||||
|
||||
def get_security_list(self, market, start):
|
||||
if start == 0:
|
||||
raise UnicodeDecodeError("gbk", b"\xbb", 0, 1, "broken")
|
||||
if start == 1000:
|
||||
return [{"code": "000001", "name": "平安银行"}]
|
||||
return []
|
||||
|
||||
@contextmanager
|
||||
def fake_connected_session():
|
||||
yield FakeApi()
|
||||
|
||||
monkeypatch.setattr(tdx_daily, "connected_session", fake_connected_session)
|
||||
|
||||
rows = tdx_daily.fetch_stock_list_rows()
|
||||
|
||||
assert rows == [{"code": "000001", "name": "平安银行", "ipo_date": None}]
|
||||
|
||||
|
||||
def test_connected_session_preserves_api_call_error(monkeypatch):
|
||||
class FakeApi:
|
||||
def __init__(self, *args, **kwargs):
|
||||
self.disconnected = False
|
||||
|
||||
def connect(self, host, port, time_out):
|
||||
return True
|
||||
|
||||
def disconnect(self):
|
||||
self.disconnected = True
|
||||
|
||||
monkeypatch.setattr(tdx_daily, "_iter_hosts", lambda: [("127.0.0.1", 7709)])
|
||||
monkeypatch.setattr(tdx_daily, "_tdx_timeout", lambda: 1)
|
||||
monkeypatch.setitem(__import__("sys").modules, "pytdx.hq", type("FakeModule", (), {"TdxHq_API": FakeApi}))
|
||||
|
||||
try:
|
||||
with tdx_daily.connected_session():
|
||||
raise UnicodeDecodeError("gbk", b"\xbb", 0, 1, "broken")
|
||||
except UnicodeDecodeError:
|
||||
pass
|
||||
else:
|
||||
raise AssertionError("expected original UnicodeDecodeError")
|
||||
|
||||
|
||||
def test_fetch_trading_days_uses_index_bars(monkeypatch):
|
||||
calls = []
|
||||
|
||||
class FakeApi:
|
||||
def get_index_bars(self, category, market, code, start, count):
|
||||
calls.append((category, market, code, start, count))
|
||||
return [{"datetime": "2026-06-19"}] if start == 0 else []
|
||||
|
||||
def get_security_bars(self, *args, **kwargs):
|
||||
raise AssertionError("fetch_trading_days should use get_index_bars")
|
||||
|
||||
@contextmanager
|
||||
def fake_connected_session():
|
||||
yield FakeApi()
|
||||
|
||||
monkeypatch.setattr(tdx_daily, "connected_session", fake_connected_session)
|
||||
monkeypatch.setattr(tdx_daily, "batch_upsert", lambda *args, **kwargs: None)
|
||||
|
||||
days = tdx_daily.fetch_trading_days("20260619", "20260621")
|
||||
|
||||
assert days == ["2026-06-19"]
|
||||
assert calls[0][:3] == (tdx_daily.TDX_DAILY_CATEGORY, 1, "000001")
|
||||
@@ -0,0 +1,332 @@
|
||||
"""验证 5 分钟数据抓取。"""
|
||||
|
||||
from contextlib import contextmanager
|
||||
from datetime import datetime
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from src.fetchers import tdx_minute
|
||||
|
||||
|
||||
def test_minute_category_mapping_only_supports_five_minutes():
|
||||
assert tdx_minute.minute_frequency_to_category("5m") == 0
|
||||
with pytest.raises(ValueError, match="仅支持 5m"):
|
||||
tdx_minute.minute_frequency_to_category("15m")
|
||||
|
||||
|
||||
def test_minute_bars_to_rows_converts_and_filters_dates():
|
||||
bars = [
|
||||
{
|
||||
"datetime": "2026-05-16 09:35",
|
||||
"open": 10,
|
||||
"high": 10.5,
|
||||
"low": 9.9,
|
||||
"close": 10.2,
|
||||
"vol": 100,
|
||||
"amount": 1020,
|
||||
},
|
||||
{
|
||||
"datetime": "2026-05-17 09:35",
|
||||
"open": "10.2",
|
||||
"high": "10.8",
|
||||
"low": "10.0",
|
||||
"close": "10.6",
|
||||
"vol": "120",
|
||||
"amount": "1272",
|
||||
},
|
||||
]
|
||||
|
||||
rows = tdx_minute._tdx_minute_bars_to_rows(
|
||||
"sh.600000",
|
||||
bars,
|
||||
start_date="20260517",
|
||||
end_date="20260517",
|
||||
)
|
||||
|
||||
assert rows == [
|
||||
{
|
||||
"code": "600000",
|
||||
"datetime": datetime(2026, 5, 17, 9, 35),
|
||||
"open": 10.2,
|
||||
"high": 10.8,
|
||||
"low": 10.0,
|
||||
"close": 10.6,
|
||||
"volume": 120.0,
|
||||
"amount": 1272.0,
|
||||
}
|
||||
]
|
||||
|
||||
|
||||
def test_fetch_one_stock_minute_uses_expected_category(monkeypatch):
|
||||
calls = []
|
||||
|
||||
class FakeApi:
|
||||
def get_security_bars(self, category, market, code, start, count):
|
||||
calls.append((category, market, code, start, count))
|
||||
if start == 0:
|
||||
return [
|
||||
{
|
||||
"datetime": "2026-05-17 09:35",
|
||||
"open": 10,
|
||||
"high": 11,
|
||||
"low": 9,
|
||||
"close": 10.5,
|
||||
"vol": 100,
|
||||
"amount": 1050,
|
||||
}
|
||||
]
|
||||
return []
|
||||
|
||||
@contextmanager
|
||||
def fake_connected_session():
|
||||
yield FakeApi()
|
||||
|
||||
monkeypatch.setattr(tdx_minute.tdx_daily, "connected_session", fake_connected_session)
|
||||
|
||||
rows = tdx_minute.fetch_one_stock_minute("600000", "20260517", "20260517", freq=5)
|
||||
|
||||
assert calls[0][:3] == (0, 1, "600000")
|
||||
assert rows[0]["code"] == "600000"
|
||||
assert rows[0]["datetime"].strftime("%Y-%m-%d %H:%M:%S") == "2026-05-17 09:35:00"
|
||||
|
||||
|
||||
def test_fetch_index_min5_uses_index_endpoint(monkeypatch):
|
||||
calls = []
|
||||
|
||||
class FakeApi:
|
||||
def get_index_bars(self, category, market, code, start, count):
|
||||
calls.append((category, market, code, start, count))
|
||||
if start == 0:
|
||||
return [
|
||||
{
|
||||
"datetime": "2026-05-17 09:35",
|
||||
"open": 3300,
|
||||
"high": 3310,
|
||||
"low": 3290,
|
||||
"close": 3305,
|
||||
"vol": 100,
|
||||
"amount": 1050,
|
||||
}
|
||||
]
|
||||
return []
|
||||
|
||||
@contextmanager
|
||||
def fake_connected_session():
|
||||
yield FakeApi()
|
||||
|
||||
monkeypatch.setattr(tdx_minute.tdx_daily, "connected_session", fake_connected_session)
|
||||
monkeypatch.setattr(tdx_minute, "get_session", lambda: type("Session", (), {
|
||||
"execute": lambda self, statement: [], "close": lambda self: None,
|
||||
})())
|
||||
saved = []
|
||||
monkeypatch.setattr(tdx_minute, "batch_upsert", lambda model, rows, keys: saved.extend(rows))
|
||||
|
||||
count = tdx_minute.fetch_index_min5(
|
||||
"20260517",
|
||||
"20260517",
|
||||
code="000300",
|
||||
source="tdx",
|
||||
market=1,
|
||||
name="沪深300",
|
||||
trading_days=["2026-05-17"],
|
||||
)
|
||||
|
||||
assert calls[0][:3] == (0, 1, "000300")
|
||||
assert count == 1
|
||||
assert saved[0]["close"] == 3305.0
|
||||
|
||||
|
||||
def test_fetch_index_min5_uses_eastmoney_for_csi_2000(monkeypatch):
|
||||
bars = [
|
||||
{
|
||||
"datetime": "2026-05-17 09:35",
|
||||
"open": 2000,
|
||||
"high": 2010,
|
||||
"low": 1990,
|
||||
"close": 2005,
|
||||
"vol": 100,
|
||||
"amount": 1050,
|
||||
}
|
||||
]
|
||||
monkeypatch.setattr(tdx_minute, "_fetch_eastmoney_index_min5_bars", lambda code: bars)
|
||||
monkeypatch.setattr(tdx_minute, "get_session", lambda: type("Session", (), {
|
||||
"execute": lambda self, statement: [], "close": lambda self: None,
|
||||
})())
|
||||
saved = []
|
||||
monkeypatch.setattr(tdx_minute, "batch_upsert", lambda model, rows, keys: saved.extend(rows))
|
||||
|
||||
count = tdx_minute.fetch_index_min5(
|
||||
"20260517",
|
||||
"20260517",
|
||||
code="932000",
|
||||
source="eastmoney",
|
||||
market=None,
|
||||
name="中证2000",
|
||||
trading_days=["2026-05-17"],
|
||||
)
|
||||
|
||||
assert count == 1
|
||||
assert saved[0]["code"] == "932000"
|
||||
assert saved[0]["close"] == 2005.0
|
||||
|
||||
|
||||
def test_eastmoney_trends_are_aggregated_to_complete_min5_bars():
|
||||
trends = [
|
||||
"2026-07-10 09:30,99,99,99,99,1,10,99",
|
||||
"2026-07-10 09:31,100,101,102,99,10,1000,100",
|
||||
"2026-07-10 09:32,101,102,103,100,20,2000,101",
|
||||
"2026-07-10 09:33,102,103,104,101,30,3000,102",
|
||||
"2026-07-10 09:34,103,104,105,102,40,4000,103",
|
||||
"2026-07-10 09:35,104,105,106,103,50,5000,104",
|
||||
"2026-07-10 09:36,105,106,107,104,60,6000,105",
|
||||
]
|
||||
|
||||
bars = tdx_minute._eastmoney_trends_to_min5_bars(trends)
|
||||
|
||||
assert bars == [{
|
||||
"datetime": datetime(2026, 7, 10, 9, 35),
|
||||
"open": 100.0,
|
||||
"close": 105.0,
|
||||
"high": 106.0,
|
||||
"low": 99.0,
|
||||
"vol": 150.0,
|
||||
"amount": 15000.0,
|
||||
}]
|
||||
|
||||
|
||||
def test_fetch_eastmoney_index_min5_uses_trends_endpoint(monkeypatch):
|
||||
class FakeResponse:
|
||||
def raise_for_status(self):
|
||||
return None
|
||||
|
||||
def json(self):
|
||||
return {
|
||||
"data": {
|
||||
"code": "932000",
|
||||
"trends": [
|
||||
f"2026-07-10 09:{minute:02d},100,101,102,99,10,1000,100"
|
||||
for minute in range(31, 36)
|
||||
],
|
||||
}
|
||||
}
|
||||
|
||||
calls = []
|
||||
|
||||
def fake_get(url, **kwargs):
|
||||
calls.append((url, kwargs))
|
||||
return FakeResponse()
|
||||
|
||||
import requests
|
||||
monkeypatch.setattr(requests, "get", fake_get)
|
||||
|
||||
bars = tdx_minute._fetch_eastmoney_index_min5_bars("932000", attempts=1)
|
||||
|
||||
assert calls[0][0].endswith("/api/qt/stock/trends2/get")
|
||||
assert calls[0][1]["params"]["secid"] == "2.932000"
|
||||
assert bars[0]["datetime"] == datetime(2026, 7, 10, 9, 35)
|
||||
|
||||
|
||||
def test_fetch_market_indices_passes_each_index_source(monkeypatch):
|
||||
calls = []
|
||||
monkeypatch.setattr(
|
||||
tdx_minute,
|
||||
"fetch_index_min5",
|
||||
lambda start_date, end_date, **kwargs: calls.append((start_date, end_date, kwargs)) or 48,
|
||||
)
|
||||
|
||||
count = tdx_minute.fetch_market_indices_min5(
|
||||
"20260517",
|
||||
"20260517",
|
||||
trading_days=["2026-05-17"],
|
||||
)
|
||||
|
||||
assert count == 384
|
||||
assert [(call[2]["code"], call[2]["source"]) for call in calls] == [
|
||||
("000001", "tdx"),
|
||||
("000016", "tdx"),
|
||||
("000300", "tdx"),
|
||||
("000905", "tdx"),
|
||||
("000852", "tdx"),
|
||||
("932000", "eastmoney"),
|
||||
("000688", "tdx"),
|
||||
("399006", "tdx"),
|
||||
]
|
||||
|
||||
|
||||
def test_fetch_minute_writes_stock_min5():
|
||||
rows = [
|
||||
{
|
||||
"code": "600519",
|
||||
"datetime": datetime(2026, 5, 17, 9, 45),
|
||||
"open": 10,
|
||||
"high": 11,
|
||||
"low": 9,
|
||||
"close": 10.5,
|
||||
"volume": 100,
|
||||
"amount": 1050,
|
||||
}
|
||||
]
|
||||
|
||||
with patch.object(tdx_minute, "get_fetch_config", return_value={"delay": 0}), \
|
||||
patch.object(tdx_minute, "_load_trading_days", return_value=["2026-05-17"]), \
|
||||
patch.object(tdx_minute, "_get_existing_minute_days", return_value={}), \
|
||||
patch.object(tdx_minute, "_get_latest_minute_dates", return_value={}), \
|
||||
patch.object(tdx_minute, "get_suspended_trade_days", return_value={}), \
|
||||
patch.object(tdx_minute, "_fetch_one_stock_minute_with_api_retry", return_value=rows), \
|
||||
patch.object(tdx_minute, "fetch_market_indices_min5"), \
|
||||
patch.object(tdx_minute.tdx_daily, "connected_session") as mock_connected, \
|
||||
patch.object(tdx_minute, "batch_upsert") as mock_upsert, \
|
||||
patch.object(tdx_minute.time, "sleep", return_value=None):
|
||||
mock_connected.return_value.__enter__.return_value = object()
|
||||
tdx_minute.fetch_tdx_minute(
|
||||
symbol="600519",
|
||||
start_date="20260517",
|
||||
end_date="20260517",
|
||||
limit=None,
|
||||
)
|
||||
|
||||
mock_upsert.assert_called_once()
|
||||
model_cls, saved_rows, index_columns = mock_upsert.call_args.args
|
||||
assert model_cls.__tablename__ == "stock_min5"
|
||||
assert saved_rows == rows
|
||||
assert index_columns == ["code", "datetime"]
|
||||
|
||||
|
||||
def test_fetch_minute_limit_caps_pending_symbols():
|
||||
calls = []
|
||||
|
||||
def fake_fetch(api, code, *args, **kwargs):
|
||||
calls.append(code)
|
||||
return [
|
||||
{
|
||||
"code": code,
|
||||
"datetime": datetime(2026, 5, 17, 10, 30),
|
||||
"open": 10,
|
||||
"high": 11,
|
||||
"low": 9,
|
||||
"close": 10.5,
|
||||
"volume": 100,
|
||||
"amount": 1050,
|
||||
}
|
||||
]
|
||||
|
||||
with patch.object(tdx_minute, "get_fetch_config", return_value={"delay": 0}), \
|
||||
patch.object(tdx_minute, "get_stock_codes", return_value=["600001", "600002", "600003"]), \
|
||||
patch.object(tdx_minute, "_load_trading_days", return_value=["2026-05-17"]), \
|
||||
patch.object(tdx_minute, "_get_existing_minute_days", return_value={}), \
|
||||
patch.object(tdx_minute, "_get_latest_minute_dates", return_value={}), \
|
||||
patch.object(tdx_minute, "get_suspended_trade_days", return_value={}), \
|
||||
patch.object(tdx_minute, "_fetch_one_stock_minute_with_api_retry", side_effect=fake_fetch), \
|
||||
patch.object(tdx_minute, "fetch_market_indices_min5"), \
|
||||
patch.object(tdx_minute.tdx_daily, "connected_session") as mock_connected, \
|
||||
patch.object(tdx_minute, "batch_upsert"), \
|
||||
patch.object(tdx_minute.time, "sleep", return_value=None):
|
||||
mock_connected.return_value.__enter__.return_value = object()
|
||||
tdx_minute.fetch_tdx_minute(
|
||||
start_date="20260517",
|
||||
end_date="20260517",
|
||||
limit=2,
|
||||
)
|
||||
|
||||
assert calls == ["600001", "600002"]
|
||||
@@ -0,0 +1,54 @@
|
||||
"""验证交易状态归档逻辑。"""
|
||||
|
||||
from datetime import date, datetime
|
||||
|
||||
from src.fetchers import trade_status
|
||||
|
||||
|
||||
def test_daily_supported_code_excludes_indexes_and_unsupported_markets():
|
||||
assert trade_status._is_daily_supported_code("600000") is True
|
||||
assert trade_status._is_daily_supported_code("300001") is True
|
||||
assert trade_status._is_daily_supported_code("399001") is False
|
||||
assert trade_status._is_daily_supported_code("920000") is False
|
||||
|
||||
|
||||
def test_build_trade_status_rows_classifies_trading_suspended_and_not_listed():
|
||||
rows = trade_status._build_trade_status_rows(
|
||||
day=date(2026, 6, 18),
|
||||
stocks={
|
||||
"000001": date(1991, 4, 3),
|
||||
"000002": date(1991, 1, 29),
|
||||
"600001": date(2026, 7, 1),
|
||||
},
|
||||
daily_codes={"000001"},
|
||||
minute_counts={
|
||||
5: {"000001": 48, "000002": 0},
|
||||
},
|
||||
coverage_threshold=0.5,
|
||||
checked_at=datetime(2026, 6, 18, 16, 0),
|
||||
)
|
||||
|
||||
by_code = {row["code"]: row for row in rows}
|
||||
assert by_code["000001"]["status"] == trade_status.STATUS_TRADING
|
||||
assert by_code["000001"]["source"] == "market"
|
||||
assert by_code["000001"]["daily_exists"] == 1
|
||||
assert by_code["000001"]["min5_count"] == 48
|
||||
assert by_code["000002"]["status"] == trade_status.STATUS_SUSPENDED
|
||||
assert by_code["000002"]["reason"] == "listed_without_daily_bar"
|
||||
assert by_code["600001"]["status"] == trade_status.STATUS_NOT_LISTED
|
||||
|
||||
|
||||
def test_build_trade_status_rows_marks_source_error_when_market_coverage_low():
|
||||
rows = trade_status._build_trade_status_rows(
|
||||
day=date(2026, 6, 18),
|
||||
stocks={
|
||||
"000001": date(1991, 4, 3),
|
||||
"000002": date(1991, 1, 29),
|
||||
},
|
||||
daily_codes=set(),
|
||||
minute_counts={},
|
||||
coverage_threshold=0.8,
|
||||
checked_at=datetime(2026, 6, 18, 16, 0),
|
||||
)
|
||||
|
||||
assert {row["status"] for row in rows} == {trade_status.STATUS_SOURCE_ERROR}
|
||||
@@ -0,0 +1,52 @@
|
||||
"""验证交易日历兼容入口。"""
|
||||
|
||||
from types import SimpleNamespace
|
||||
from unittest.mock import patch
|
||||
|
||||
from src.fetchers import trading_day
|
||||
|
||||
|
||||
def test_fetch_trading_days_delegates_to_current_fetcher():
|
||||
with patch.object(trading_day.tdx_daily, "fetch_trading_days", return_value=["2026-05-08"]) as mock_fetch:
|
||||
days = trading_day.fetch_trading_days(start_date="20260508", end_date="20260511")
|
||||
|
||||
assert days == ["2026-05-08"]
|
||||
mock_fetch.assert_called_once_with("20260508", "20260511")
|
||||
|
||||
|
||||
def test_get_trading_days_skips_refresh_when_calendar_loaded_past_end_date():
|
||||
calls = {"count": 0}
|
||||
|
||||
def fake_execute(*args, **kwargs):
|
||||
calls["count"] += 1
|
||||
if calls["count"] == 1:
|
||||
return [("2026-05-11",), ("2026-05-12",)]
|
||||
return SimpleNamespace(scalar=lambda: "2026-12-31")
|
||||
|
||||
fake_session = SimpleNamespace(execute=fake_execute, close=lambda: None)
|
||||
|
||||
with patch.object(trading_day, "get_session", return_value=fake_session), \
|
||||
patch.object(trading_day.tdx_daily, "fetch_trading_days") as mock_fetch:
|
||||
days = trading_day.get_trading_days("20260510", "20260512")
|
||||
|
||||
assert days == ["2026-05-11", "2026-05-12"]
|
||||
mock_fetch.assert_not_called()
|
||||
|
||||
|
||||
def test_get_trading_days_refreshes_when_local_calendar_incomplete():
|
||||
calls = {"count": 0}
|
||||
|
||||
def fake_execute(*args, **kwargs):
|
||||
calls["count"] += 1
|
||||
if calls["count"] == 1:
|
||||
return [("2026-05-11",)]
|
||||
return SimpleNamespace(scalar=lambda: "2026-05-11")
|
||||
|
||||
fake_session = SimpleNamespace(execute=fake_execute, close=lambda: None)
|
||||
|
||||
with patch.object(trading_day, "get_session", return_value=fake_session), \
|
||||
patch.object(trading_day.tdx_daily, "fetch_trading_days", return_value=["2026-05-12"]) as mock_fetch:
|
||||
days = trading_day.get_trading_days("20260510", "20260512")
|
||||
|
||||
assert days == ["2026-05-11", "2026-05-12"]
|
||||
mock_fetch.assert_called_once_with("20260510", "20260512")
|
||||
Reference in New Issue
Block a user