Framwork Architecture

The proposed system integrates a pretrained graph neural network (GNN) with a large language model (LLM) to handle both predefined and open-ended tasks. The architecture consists of four primary modules:

• Frozen Graph Encoder (FGE): A pretrained GNN, such as GraphSAGE, that generates dense vector representations for nodes in a large-scale graph.
• Frozen Text Generator (FTG): A pretrained LLM, like ChatGLM2-6B, capable of understanding and generating human-like text.
• Data Synthesizer: A component that constructs aligned data pairs, combining node embeddings with textual descriptions.
• Embedding Projector: A module that maps node embeddings from the GNN space into the token space of the LLM, bridging the modality gap.

Training Procedure

The training is divided into two distinct stages:

Stage 1: Graph-Text Alignment

The Embedding Projector is trained to align GNN node embeddings with textual data. The FGE generates node vectors. The Data Synthesizer uses the FTG to create question-answer pairs based on node attribtues. The projector learns to transform node embeddings so that they are contextually relevant to the synthesized text, minimizing the discrepancy between the projected embedding and the text representation.

Stage 2: Graph-LLM Integration

The output from the Embedding Projector is combined with instructional prompts and fed into the FTG. The model is trained to generate accurate responses by aligning its predictions with ground-truth answers, effectively teaching the LLM to reason over graph-structured data.

Experimental Setup

Evaluation was conducted on two real-world datasets:

• E-commerce Network: A subset from an online retail platform with 980,000 user nodes and 1.79 million social edges. Attributes include user interactions like purchases and browsing history.
• Academic Citation Graph: Built from arXiv papers, containing 169,343 nodes (papers) and 1,166,243 citation edges. Each node includes the paper's title and abstract.

Implementation Guide

Environment Setup

conda create -n gnn_llm_align python=3.9
conda activate gnn_llm_align
git clone https://example.com/gnn-llm-align.git
cd gnn-llm-align/
pip install -r deps.txt

Data and Model Preparation

Place the required dataset files (node_embeddings.pt, graph_structure.pt, metadata.tsv) into the ./data/citation/ directory. Download the pretrained LLM checkpoint and place it in ./models/text_generator/.

Execution Phases

Data Synthesis

Generate textual summaries for nodes using the LLM.

cd ./synthesis/
python generate_summaries.py

Model Training

Alignment Stage 1

Train the projector for graph-text alignment.

cd ./training/
python run_training.py --config stage1_citation.yaml

Integration Stage 2

Fine-tune the entire system for graph-LLM integration.

cd ./training/
python run_training.py --config stage2_citation.yaml

Inference

Generate predictions using the trained model.

cd ./inference/
python predict.py --config inference_citation.yaml

Performance Assessmant

Evaluate the prediction accuracy.

cd ./evaluation/
python calculate_metrics.py