Compare commits

...

4 Commits

Author SHA1 Message Date
arvinxx 1e8031066c fix 2025-08-12 11:46:19 +08:00
arvinxx 84bab1008b add web search config 2025-08-12 11:45:00 +08:00
arvinxx 8f80c0e40b update 2025-08-12 11:44:49 +08:00
arvinxx cb243e0867 add 2025-08-12 11:44:48 +08:00
15 changed files with 725 additions and 4 deletions
+250
View File
@@ -0,0 +1,250 @@
# CLAUDE.md
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
## Project Overview
LobeChat is an open-source, modern-design AI chat framework built with Next.js 15. It supports multi-modal AI interactions, extensible plugin systems, and one-click deployment. The project emphasizes design engineering, performance, and developer experience.
**Key Technologies:**
- Next.js 15 with App Router
- React 19 with Server Components
- TypeScript throughout
- Zustand for state management
- tRPC for type-safe APIs
- Drizzle ORM with PostgreSQL/PGlite
- Ant Design + @lobehub/ui components
- pnpm as package manager
## Development Commands
### Core Development
```bash
# Start development server
pnpm dev # Port 3010 with Turbopack
pnpm dev:desktop # Desktop development on port 3015
# Build and deployment
pnpm build # Production build
pnpm build:analyze # Bundle analysis
pnpm build:docker # Docker build
pnpm start # Start production server (port 3210)
# Desktop application
pnpm desktop:build # Full desktop build pipeline
```
### Database Operations
```bash
# Schema and migrations
pnpm db:generate # Generate Drizzle schema + client migrations
pnpm db:migrate # Run server database migrations
# Development database
pnpm db:push-test # Push to test database
```
### Code Quality
```bash
# Linting and type checking
pnpm lint # Run all linting (TS, style, circular deps)
pnpm lint:ts # ESLint for TS/JS files
pnpm lint:style # Stylelint for styled components
pnpm type-check # TypeScript compiler check
# Testing
pnpm test # Run all tests (app + server)
pnpm test-app # Client-side tests only
pnpm test-server # Server-side tests only
```
### Content Workflows
```bash
# Internationalization
pnpm i18n # Generate and process translations
pnpm docs:i18n # Process documentation translations
# Documentation
pnpm workflow:docs # Generate documentation workflows
pnpm workflow:mdx # Process MDX files
```
## Architecture Overview
### Multi-Environment Database Strategy
The application supports three deployment modes:
- **Browser/PWA**: PGlite (WASM PostgreSQL) for local data
- **Server**: Remote PostgreSQL with full multi-user support
- **Desktop**: PGlite with optional cloud sync via tRPC
### State Management (Zustand Slices)
Located in `src/store/`, organized by domain:
- `chat/` - Messages, AI interactions, tools, topics
- `session/` - Session management and switching
- `user/` - Settings, preferences, authentication
- `agent/` - AI agent configurations
- `aiInfra/` - Model providers and infrastructure
- `file/` - File upload and knowledge base
- `tool/` - Plugin and tool management
Each slice follows the pattern:
```
slices/[domain]/
├── actions/ (or action.ts) # State mutations
├── initialState.ts # State interface + defaults
├── selectors.ts # State queries (export as xxxSelectors)
└── reducer.ts (optional) # Immer-based reducers
```
### Backend Architecture (src/server/)
Three-layer backend design:
1. **tRPC Routers** (`src/server/routers/`) - API endpoints
2. **Service Layer** (`src/server/services/`) - Business logic with platform abstractions
3. **Data Layer** (`src/database/`) - Models, repositories, schemas
Services use `impls/` subdirectories to abstract platform differences (e.g., S3 vs local storage).
### Frontend Architecture (src/app/)
Next.js App Router with advanced patterns:
- **Parallel Routes**: `@modal`, `@session`, `@conversation`, `@portal`
- **Route Groups**: `(backend)/` for APIs, `[variants]/` for app routes
- **Layout Strategy**: Responsive desktop/mobile layouts throughout
- **Server Components**: Extensive use for performance
### Feature Organization (src/features/)
Domain-driven feature modules with co-located:
- Components and UI
- Hooks and utilities
- Local state management
- Feature-specific services
## Key Patterns and Conventions
### Component Architecture
- Use functional components with hooks
- Prefer Server Components when possible
- Co-locate components with their features
- Use @lobehub/ui and Ant Design consistently
### State Management Patterns
- Zustand slices with selector aggregation: `export const xxxSelectors = { ... }`
- Immer for immutable updates in reducers
- Map structures for relational data: `Record<string, T[]>`
- Loading states as ID arrays: `loadingIds: string[]`
### Database Patterns
- Drizzle ORM with schema-first approach
- Repository pattern for complex queries
- Direct model access for simple CRUD
- Dual migration system (server + client)
### API Patterns
- tRPC for type-safe server communication
- Service layer abstractions with `impls/` for platform differences
- RESTful endpoints for webhooks and external integrations
### File Naming Conventions
- Use kebab-case for directories and files
- React components in PascalCase
- TypeScript interfaces with descriptive names
- Consistent patterns: `initialState.ts`, `selectors.ts`, `action.ts`
## AI Provider Integration
The project supports 42+ AI providers through a unified abstraction layer. When adding new providers:
1. Implement in `src/libs/agent-runtime/`
2. Add configuration in `src/config/aiModels/`
3. Update provider lists and documentation
4. Test with multiple model types (text, vision, reasoning)
## Testing Strategy
- **Unit Tests**: Vitest for utilities and services
- **Component Tests**: React Testing Library for UI
- **Integration Tests**: Full API and database flows
- **Database Tests**: Separate test database configuration
Run specific test suites:
```bash
pnpm test-app:coverage # Client tests with coverage
pnpm test-server:coverage # Server tests with coverage
```
## Deployment Notes
### Environment Variables
Critical variables include:
- `OPENAI_API_KEY` - AI provider access
- `DATABASE_URL` - PostgreSQL connection (server mode)
- `NEXTAUTH_SECRET` - Authentication secret
- Provider-specific API keys
### Docker Deployment
```bash
# Standard build
pnpm self-hosting:docker
# With Chinese mirror
pnpm self-hosting:docker-cn
# Database variant
pnpm self-hosting:docker-cn@database
```
### Desktop Application
The Electron app requires special build processes:
```bash
pnpm desktop:build-next # Next.js build for desktop
pnpm desktop:prepare-dist # Prepare distribution files
pnpm desktop:build-electron # Electron packaging
```
## Performance Considerations
- Lighthouse scores target >90 for all metrics
- Use React Server Components for initial page loads
- Implement proper loading states and skeleton UIs
- Optimize bundle size with dynamic imports
- Leverage Turbopack for fast development builds
## Contributing Guidelines
When working on this codebase:
1. Follow the established architectural patterns
2. Use TypeScript strictly - avoid `any` types
3. Update tests for new functionality
4. Run the full lint suite before committing
5. Consider both local and server deployment modes
6. Test with multiple AI providers when relevant
7. Maintain responsive design across desktop/mobile
This architecture enables LobeChat to scale from personal use to enterprise deployment while maintaining excellent developer experience and performance.
@@ -4,6 +4,7 @@ import { UserGeneralConfig } from './general';
import { UserHotkeyConfig } from './hotkey';
import { UserKeyVaults } from './keyVaults';
import { UserModelProviderConfig } from './modelProvider';
import { UserSearchConfig } from './search';
import { UserSyncSettings } from './sync';
import { UserSystemAgentConfig } from './systemAgent';
import { UserToolConfig } from './tool';
@@ -15,6 +16,7 @@ export * from './general';
export * from './hotkey';
export * from './keyVaults';
export * from './modelProvider';
export * from './search';
export * from './sync';
export * from './systemAgent';
export * from './tts';
@@ -28,6 +30,7 @@ export interface UserSettings {
hotkey: UserHotkeyConfig;
keyVaults: UserKeyVaults;
languageModel: UserModelProviderConfig;
search: UserSearchConfig;
sync?: UserSyncSettings;
systemAgent: UserSystemAgentConfig;
tool: UserToolConfig;
@@ -0,0 +1,34 @@
export interface UserSearchConfig {
crawlApiKey?: string;
/**
* 爬虫配置
*/
crawlConfig?: {
executeJS?: boolean;
includeImages?: boolean;
maxContentLength?: number;
outputFormat?: 'markdown' | 'html' | 'text';
timeout?: number;
};
crawlEndpoint?: string;
/**
* 爬虫提供商配置
*/
crawlProvider?: string;
searchApiKey?: string;
/**
* 搜索引擎配置
*/
searchConfig?: {
excludeDomains?: string;
maxResults?: number;
};
searchEndpoint?: string;
/**
* 搜索提供商配置
*/
searchProvider?: string;
}
@@ -5,6 +5,7 @@ import {
Cloudy,
Database,
EthernetPort,
Globe,
Info,
KeyboardIcon,
Mic2,
@@ -88,7 +89,7 @@ export const useCategory = () => {
),
}
: {
icon: <Icon icon={Brain} />,
icon: <Icon icon={Cloudy} />,
key: SettingsTabs.Provider,
label: (
<Link href={'/settings/provider'} onClick={(e) => e.preventDefault()}>
@@ -96,7 +97,15 @@ export const useCategory = () => {
</Link>
),
}),
{
icon: <Icon icon={Globe} />,
key: SettingsTabs.Search,
label: (
<Link href={'/settings/search'} onClick={(e) => e.preventDefault()}>
{t('tab.search')}
</Link>
),
},
enableSTT && {
icon: <Icon icon={Mic2} />,
key: SettingsTabs.TTS,
@@ -115,6 +124,7 @@ export const useCategory = () => {
</Link>
),
},
{
type: 'divider',
},
@@ -0,0 +1,31 @@
import { CrawlProvider, SearchProvider } from './types';
export const searchProviders: SearchProvider[] = [
{
description: 'AI search engine focused on research and analysis',
id: 'tavily',
name: 'Tavily',
},
{ description: 'Web crawling and search service', id: 'firecrawl', name: 'Firecrawl' },
{ description: 'Multimodal search engine', id: 'jina', name: 'Jina' },
{ description: 'Semantic search engine', id: 'exa', name: 'Exa' },
{ description: 'Chinese-optimized search engine', id: 'bocha', name: 'Bocha' },
];
export const crawlProviders: CrawlProvider[] = [
{ description: 'Simple and fast web crawling', id: 'naive', name: 'Naive', needsApi: false },
{ description: 'Intelligent content extraction', id: 'jina', name: 'Jina', needsApi: true },
{
description: 'Headless browser crawling',
id: 'browserless',
name: 'Browserless',
needsApi: true,
},
{
description: 'Semantic content crawling',
id: 'exa',
name: 'Exa',
needsApi: true,
sharedWith: 'exa',
},
];
@@ -0,0 +1,121 @@
'use client';
import { Form, type FormGroupItemType, type FormItemProps } from '@lobehub/ui';
import { Input, Select, Skeleton, Slider, Switch } from 'antd';
import isEqual from 'fast-deep-equal';
import { memo } from 'react';
import { useTranslation } from 'react-i18next';
import { FORM_STYLE } from '@/const/layoutTokens';
import { useUserStore } from '@/store/user';
import { settingsSelectors } from '@/store/user/selectors';
import { crawlProviders } from '../constants';
const CrawlProvider = memo(() => {
const { t } = useTranslation('setting');
const [form] = Form.useForm();
const settings = useUserStore(settingsSelectors.currentSettings, isEqual);
const [setSettings, isUserStateInit] = useUserStore((s) => [s.setSettings, s.isUserStateInit]);
if (!isUserStateInit) return <Skeleton active paragraph={{ rows: 5 }} title={false} />;
const crawlProviderItems: FormItemProps[] = [
{
children: (
<Select
options={crawlProviders.map((provider) => ({
label: provider.name,
value: provider.id,
}))}
placeholder={t('search.service.crawl.desc')}
/>
),
desc: t('search.service.crawl.desc'),
label: t('search.service.crawl.title'),
name: ['search', 'crawlProvider'],
},
{
children: <Input.Password placeholder={t('search.apiKey.placeholder')} />,
desc: t('search.apiKey.desc'),
label: t('search.apiKey.title'),
name: ['search', 'crawlApiKey'],
},
{
children: <Input placeholder={t('search.endpoint.placeholder')} />,
desc: t('search.endpoint.desc'),
label: t('search.endpoint.title'),
name: ['search', 'crawlEndpoint'],
},
];
const crawlConfigItems: FormItemProps[] = [
{
children: <Slider max={120} min={5} step={5} />,
desc: t('search.crawl.timeout.desc'),
label: t('search.crawl.timeout.title'),
name: ['search', 'crawlConfig', 'timeout'],
},
{
children: (
<Select
options={[
{ label: 'Markdown', value: 'markdown' },
{ label: 'HTML', value: 'html' },
{ label: t('search.crawl.outputFormat.text'), value: 'text' },
]}
/>
),
desc: t('search.crawl.outputFormat.desc'),
label: t('search.crawl.outputFormat.title'),
name: ['search', 'crawlConfig', 'outputFormat'],
},
{
children: <Slider max={20_000} min={1000} step={500} />,
desc: t('search.crawl.maxContentLength.desc'),
label: t('search.crawl.maxContentLength.title'),
name: ['search', 'crawlConfig', 'maxContentLength'],
},
{
children: <Switch />,
desc: t('search.crawl.executeJS.desc'),
label: t('search.crawl.executeJS.title'),
name: ['search', 'crawlConfig', 'executeJS'],
valuePropName: 'checked',
},
{
children: <Switch />,
desc: t('search.crawl.includeImages.desc'),
label: t('search.crawl.includeImages.title'),
name: ['search', 'crawlConfig', 'includeImages'],
valuePropName: 'checked',
},
];
const crawlProviderGroup: FormGroupItemType = {
children: crawlProviderItems,
title: t('search.service.title'),
};
const crawlConfigGroup: FormGroupItemType = {
children: crawlConfigItems,
title: t('search.crawl.title'),
};
return (
<Form
form={form}
initialValues={settings}
items={[crawlProviderGroup, crawlConfigGroup]}
itemsType={'group'}
onValuesChange={setSettings}
variant={'borderless'}
{...FORM_STYLE}
/>
);
});
CrawlProvider.displayName = 'CrawlProvider';
export default CrawlProvider;
@@ -0,0 +1,95 @@
'use client';
import { Form, type FormGroupItemType, type FormItemProps } from '@lobehub/ui';
import { Input, Select, Skeleton, Slider } from 'antd';
import isEqual from 'fast-deep-equal';
import { memo } from 'react';
import { useTranslation } from 'react-i18next';
import { FORM_STYLE } from '@/const/layoutTokens';
import { useUserStore } from '@/store/user';
import { settingsSelectors } from '@/store/user/selectors';
import { searchProviders } from '../constants';
const SearchProvider = memo(() => {
const { t } = useTranslation('setting');
const [form] = Form.useForm();
const settings = useUserStore(settingsSelectors.currentSettings, isEqual);
const [setSettings, isUserStateInit] = useUserStore((s) => [s.setSettings, s.isUserStateInit]);
if (!isUserStateInit) return <Skeleton active paragraph={{ rows: 5 }} title={false} />;
const searchProviderItems: FormItemProps[] = [
{
children: (
<Select
options={searchProviders.map((provider) => ({
label: provider.name,
value: provider.id,
}))}
placeholder={t('search.service.search.desc')}
/>
),
desc: t('search.service.search.desc'),
label: t('search.service.search.title'),
name: ['search', 'searchProvider'],
},
{
children: <Input.Password placeholder={t('search.apiKey.placeholder')} />,
desc: t('search.apiKey.desc'),
label: t('search.apiKey.title'),
name: ['search', 'searchApiKey'],
},
{
children: <Input placeholder={t('search.endpoint.placeholder')} />,
desc: t('search.endpoint.desc'),
label: t('search.endpoint.title'),
name: ['search', 'searchEndpoint'],
},
];
const searchConfigItems: FormItemProps[] = [
{
children: <Slider max={50} min={1} />,
desc: t('search.config.maxResults.desc'),
label: t('search.config.maxResults.title'),
name: ['search', 'searchConfig', 'maxResults'],
},
{
children: (
<Input.TextArea placeholder={t('search.config.excludeDomains.placeholder')} rows={3} />
),
desc: t('search.config.excludeDomains.desc'),
label: t('search.config.excludeDomains.title'),
name: ['search', 'searchConfig', 'excludeDomains'],
},
];
const searchProviderGroup: FormGroupItemType = {
children: searchProviderItems,
title: t('search.service.title'),
};
const searchConfigGroup: FormGroupItemType = {
children: searchConfigItems,
title: t('search.config.title'),
};
return (
<Form
form={form}
initialValues={settings}
items={[searchProviderGroup, searchConfigGroup]}
itemsType={'group'}
onValuesChange={setSettings}
variant={'borderless'}
{...FORM_STYLE}
/>
);
});
SearchProvider.displayName = 'SearchProvider';
export default SearchProvider;
@@ -0,0 +1,17 @@
import { Skeleton } from 'antd';
import { Flexbox } from 'react-layout-kit';
export default function Loading() {
return (
<Flexbox gap={24} style={{ margin: '0 auto', maxWidth: 800, padding: 24 }}>
<Flexbox gap={8}>
<Skeleton.Input style={{ height: 32, width: 200 }} />
<Skeleton.Input style={{ height: 20, width: 300 }} />
</Flexbox>
<Skeleton.Input style={{ height: 80, width: '100%' }} />
<Skeleton.Input style={{ height: 400, width: '100%' }} />
<Skeleton.Input style={{ height: 400, width: '100%' }} />
</Flexbox>
);
}
@@ -0,0 +1,30 @@
import { metadataModule } from '@/server/metadata';
import { translation } from '@/server/translation';
import { DynamicLayoutProps } from '@/types/next';
import { RouteVariants } from '@/utils/server/routeVariants';
import CrawlProvider from './features/CrawlProvider';
import SearchProvider from './features/SearchProvider';
export const generateMetadata = async (props: DynamicLayoutProps) => {
const locale = await RouteVariants.getLocale(props);
const { t } = await translation('setting', locale);
return metadataModule.generate({
description: t('header.desc'),
title: t('tab.search'),
url: '/settings/search',
});
};
const Page = () => {
return (
<>
<SearchProvider />
<CrawlProvider />
</>
);
};
Page.displayName = 'SearchSettingPage';
export default Page;
@@ -0,0 +1,37 @@
export interface SearchProvider {
description: string;
id: string;
name: string;
}
export interface CrawlProvider {
description: string;
id: string;
name: string;
needsApi: boolean;
sharedWith?: string;
}
export interface SearchSettings {
crawlConfig?: {
executeJS?: boolean;
includeImages?: boolean;
maxContentLength?: number;
outputFormat?: 'markdown' | 'html' | 'text';
proxy?: string;
timeout?: number;
userAgent?: string;
};
crawlProvider?: string;
searchConfig?: {
autoRetry?: boolean;
depth?: 'basic' | 'advanced';
enableCache?: boolean;
excludeDomains?: string[];
language?: string;
maxResults?: number;
};
searchProvider?: string;
}
export type TestStatus = null | 'testing' | 'success' | 'error';
+3
View File
@@ -4,6 +4,7 @@ import { DEFAULT_AGENT } from './agent';
import { DEFAULT_COMMON_SETTINGS } from './common';
import { DEFAULT_HOTKEY_CONFIG } from './hotkey';
import { DEFAULT_LLM_CONFIG } from './llm';
import { DEFAULT_SEARCH_CONFIG } from './search';
import { DEFAULT_SYNC_CONFIG } from './sync';
import { DEFAULT_SYSTEM_AGENT_CONFIG } from './systemAgent';
import { DEFAULT_TOOL_CONFIG } from './tool';
@@ -14,6 +15,7 @@ export const COOKIE_CACHE_DAYS = 30;
export * from './agent';
export * from './hotkey';
export * from './llm';
export * from './search';
export * from './systemAgent';
export * from './tool';
export * from './tts';
@@ -24,6 +26,7 @@ export const DEFAULT_SETTINGS: UserSettings = {
hotkey: DEFAULT_HOTKEY_CONFIG,
keyVaults: {},
languageModel: DEFAULT_LLM_CONFIG,
search: DEFAULT_SEARCH_CONFIG,
sync: DEFAULT_SYNC_CONFIG,
systemAgent: DEFAULT_SYSTEM_AGENT_CONFIG,
tool: DEFAULT_TOOL_CONFIG,
+24
View File
@@ -0,0 +1,24 @@
import { UserSearchConfig } from '@/types/user/settings/search';
export const DEFAULT_SEARCH_CONFIG: UserSearchConfig = {
crawlApiKey: '',
crawlConfig: {
executeJS: false,
includeImages: false,
maxContentLength: 10_000,
outputFormat: 'markdown',
timeout: 30,
},
crawlEndpoint: '',
crawlProvider: 'naive',
searchApiKey: '',
searchConfig: {
excludeDomains: '',
maxResults: 10,
},
searchEndpoint: '',
searchProvider: 'tavily',
};
+62 -2
View File
@@ -156,6 +156,67 @@ export default {
},
store: '插件商店',
},
search: {
apiKey: {
desc: '服务所需的 API 密钥,点击右侧密钥库按钮可以从密钥库中快速选择',
placeholder: '请输入 API 密钥',
title: 'API 密钥',
},
config: {
excludeDomains: {
desc: '搜索时排除的域名列表,每行一个',
placeholder: 'example.com\nbad-site.com',
title: '排除域名',
},
maxResults: {
desc: '单次搜索返回的最大结果数量',
title: '最大结果数',
},
title: '搜索配置',
},
crawl: {
executeJS: {
desc: '是否执行网页中的 JavaScript 代码',
title: '执行 JavaScript',
},
includeImages: {
desc: '是否在抓取结果中包含图片内容',
title: '包含图片',
},
maxContentLength: {
desc: '单个网页抓取的最大字符数',
title: '最大内容长度',
},
outputFormat: {
desc: '抓取内容的输出格式',
text: '纯文本',
title: '输出格式',
},
timeout: {
desc: '网页抓取的超时时间',
title: '超时时间 (秒)',
},
title: '爬虫配置',
},
endpoint: {
desc: '自定义服务的 API 端点地址,留空则使用默认地址',
placeholder: '请输入 API 端点地址',
title: 'API 端点',
},
service: {
crawl: {
desc: '选择用于网页内容抓取的服务提供商',
title: '网页爬虫',
},
desc: '管理搜索引擎和网页爬虫的服务提供商配置',
search: {
desc: '选择用于网络搜索的服务提供商',
title: '搜索引擎',
},
title: '服务提供商',
},
},
settingAgent: {
avatar: {
title: '助手头像',
@@ -184,7 +245,6 @@ export default {
},
title: '助手信息',
},
settingAppearance: {
animationMode: {
agile: '敏捷',
@@ -210,7 +270,6 @@ export default {
},
title: '应用外观',
},
settingChat: {
autoCreateTopicThreshold: {
desc: '当前消息数超过设定该值后,将自动创建话题',
@@ -552,6 +611,7 @@ export default {
'llm': '语言模型',
'provider': 'AI 服务商',
'proxy': '网络代理',
'search': '网络搜索',
'storage': '数据存储',
'sync': '云端同步',
'system-agent': '系统助手',
+1
View File
@@ -33,6 +33,7 @@ export enum SettingsTabs {
LLM = 'llm',
Provider = 'provider',
Proxy = 'proxy',
Search = 'search',
Storage = 'storage',
Sync = 'sync',
SystemAgent = 'system-agent',
+5
View File
@@ -11,6 +11,7 @@ import {
SystemAgentItem,
UserGeneralConfig,
UserKeyVaults,
UserSearchConfig,
UserSettings,
UserSystemAgentConfigKey,
} from '@/types/user/settings';
@@ -26,6 +27,7 @@ export interface UserSettingsAction {
updateDefaultAgent: (agent: PartialDeep<LobeAgentSettings>) => Promise<void>;
updateGeneralConfig: (settings: Partial<UserGeneralConfig>) => Promise<void>;
updateKeyVaults: (settings: Partial<UserKeyVaults>) => Promise<void>;
updateSearchConfig: (settings: Partial<UserSearchConfig>) => Promise<void>;
updateSystemAgent: (
key: UserSystemAgentConfigKey,
@@ -99,6 +101,9 @@ export const createSettingsSlice: StateCreator<
updateKeyVaults: async (keyVaults) => {
await get().setSettings({ keyVaults });
},
updateSearchConfig: async (search) => {
await get().setSettings({ search });
},
updateSystemAgent: async (key, value) => {
await get().setSettings({
systemAgent: { [key]: { ...value } },