Qwen3-langchain-AI Agent

1. Introduction

LangChain is a framework designed for developing large language model (LLM) applications. It supports integrating external tools and data resources and managing interaction logic, helping build complex AI applications such as intelligent dialogue and document analysis. The project's main goal is to simplify interaction with documents using natural language and to extract valuable information. This project is built with LangChain and Qwen3-7b to provide users with a smooth, natural conversational experience.

1. Features

  • Fully local inference.
  • Supports multiple document formats: PDF, DOCX, TXT.
  • Chat with document content; ask questions and get relevant answers based on the document.
  • User-friendly interface ensuring smooth interaction.

2. Project Directory

Personal demo project directory:

 |-- ChatDoc-TPU
    |-- data
        |-- db_tpu        -- 知识库持久化目录
        |-- uploaded      -- 已上传文件目录
    |-- models
        |-- bert_model    -- BERT 模型
        |-- qwen_model    -- qwen-7B 模型
    |-- chat
        |-- chatbot.py    -- ChatDoc业务逻辑脚本
        |-- charglm3      -- charglm3 代码
        |-- qwen          -- qwen 代码
    |-- embedding         -- 文本嵌入模型
    |-- docs              -- 环境安装文档
    |-- static            -- README中图片文件
    |-- README.md         -- README
    |-- config.ini        -- 推理模型配置文件
    |-- requirements.txt  -- 项目依赖
    |-- run.sh            -- 启动脚本
    |-- web_demo_st.py    -- 页面交互脚本

2. Running Steps

1. Prepare the Python Environment, Data, and Model

1.1 First confirm Python > 3.8, then create and enter the virtual environment

    sudo apt update
    #检查python版本，若不满足则需升级
    python3 --version

    # 创建虚拟环境（不含pip包）
    # ！！！每次运行都要按照步骤切换虚拟环境！！！
    cd /data
    # 创建虚拟环境（不包含 pip）
    python3.10 -m venv --without-pip myenv

    # 进入虚拟环境
    source myenv/bin/activate

1.2 Copy the Sophgo official ChatDoc-TPU project directory (or upload files to /data on the box after copying)

    git clone https://github.com/wangyifan2018/ChatDoc-TPU
    cd ChatDoc-TPU
    # 安装相关pip包
    pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple
    #检查pip包情况
    pip list

1.3 Install sail

This example depends on a new version of sail; older versions need to be updated. For the installation method, refer to Sail_Install_Guide.

2 Startup

Return to the ChatDoc-TPU main directory and start the program:

# 使用默认路径启动 Qwen-7b ，模型和配置文件自动下载
./run.sh --model qwen7b --dev\_id 0

You can also choose other models; the template is as follows:

# 当然也可以选用其他模型，参考模板如下

usage: ./run.sh [--model MODEL]  [--dev_id DEV_ID] [--server_address SERVER_ADDRESS] [--server_port SERVER_PORT]
--model: 选择模型，可选项为 chatglm3/qwen。默认为 "chatglm3"。
--dev_id: 用于推理的 TPU 设备 ID。默认为 0。
--server_address: web server 地址。默认为 "0.0.0.0"。
--server_port：web sever 端口。如不设置，从 8501 起自动分配。

3. Operation Guide

1751598696151

3.1 Interface Overview

ChatDoc consists of a control area and a chat dialog area. The control area manages documents and the knowledge base; the chat dialog area is used to enter and receive messages.

Area 10 in the figure above is the currently selected document in ChatDoc. If area 10 is empty — meaning ChatDoc has no document selected — and you still chat in the chat dialog area, ChatDoc acts as a pure LLM-based ChatBot.

3.2 Upload a Document

Click 1 to select a document to upload, then click button 4 to build the knowledge base. The document will then be embedded; once done, it will be selected and shown in area 10, and you can start chatting. You can upload documents repeatedly; all successfully embedded documents will appear in area 10.

3.3 Persist the Knowledge Base

The documents selected in area 10 are cleared when the user refreshes or closes the page. If you want to keep these embedded documents, you can persist the knowledge base; the next time you enter, the knowledge base can be loaded without re-embedding. To do so, when area 10 is not empty, click button 5 to persist the knowledge base. The knowledge base name is formed by joining all document names with commas.

3.4 Import a Knowledge Base

Users can view the currently persisted knowledge bases from dropdown 2. Select the knowledge base you want to load, then click button 3 to import it. After that, you can start chatting. Note: CPU-version knowledge bases and TPU-version knowledge bases are not interchangeable; if you launch the TPU-version program, you cannot load a persisted CPU-version knowledge base, and vice versa.

3.5 Delete a Knowledge Base

When you need to delete a locally persisted knowledge base, select it from dropdown 2, then click button 6 to delete it.

3.6 Rename a Knowledge Base

1751595366009

Because the knowledge base is named by combining its document names, the name can become long. ChatDoc provides a feature to rename a knowledge base: select the knowledge base to modify from dropdown 2, then click button 9 to rename it. ChatDoc will pop up an input box and a confirm button as shown above. Enter the new name in the input box, then click the Confirm Rename button.

3.7 Clear Chat History

Click button 7 to clear the chat history in the chat dialog area. Nothing else is affected.

3.8 Remove the Selected Document

Click button 8 to clear area 10 and the chat history at the same time.