feat: support web page crawl in the search (#6582)

* test pnpm workspace

* test bun workspace

* add crawler mode

* improve pure fetch result

* update

* improve result

* improve code

* update

* fix

* 完成 urlRules 规则集合设计

* 增加爬虫展示 ui

* 增加爬虫展示 Portal

* update

* update

* improve

* update i18n

* support view multi pages

* improve implement

* improve code

* refactor the web-browsing system prompt

* improve code

* fix open
This commit is contained in:
Arvin Xu
2025-03-02 16:39:32 +08:00
committed by GitHub
parent 8bf1b9062a
commit 5e02602457
96 changed files with 4744 additions and 255 deletions
+3
View File
@@ -0,0 +1,3 @@
packages:
- 'packages/**'
- '.'