1. 首頁 > 生活百科 > >

GNN 101( 二 )

生活百科GNN 101



GNN 101

文章插圖
  • Graph-level prediction 預測整圖或子圖的類別或性質

GNN 101

文章插圖
HowWorkflow
GNN 101

文章插圖
以fraud detection為例：
  • Tabformer數據集
    GNN 101

    文章插圖
  • workflow
    GNN 101

    文章插圖
軟件棧
GNN 101

文章插圖
  • 計算平面

GNN 101

文章插圖
  • 【GNN 101】數據平面

GNN 101

文章插圖
SW ChallengesGraph SamplerFor many small graphs datasets, full batch training works most time. Full batch training means we can do training on whole graph; When it comes to one large graph datasets, in many real scenarios, we meet Neighbor Explosion problem;
Neighbor Explosion:
GNN 101

文章插圖
Graph sampler comes to rescue. Only sample a fraction of target nodes, and furthermore, for each target node, we sample a sub-graph of its ego-network for training.This is called mini-batch training. Graph sampling is triggered for each data loading.And the hops of the sampled graph equals the GNN layer number . Which means graph sampler in data loader is important in GNN training.
GNN 101

文章插圖
Challenge: How to optimize sampler both as standalone and in training pipe?
When graph comes to huge(billions of nodes, tens of billions of edges), we meet new at-scale challenges:
  • How to store the huge graph across node? -> graph partition
  • How to build a training system w/ not only distributed model computing but also distributed graph store and sampling?
    • How to cut the graph while minimize cross partition connections?

GNN 101

文章插圖
A possible GNN distributed training architecture:
GNN 101

文章插圖
Scatter-Gather
  • Fuse adjacent graphs ops
    One common fuse pattern for GCN & GraphSAGE：
    GNN 101

    文章插圖
    Challenge: How to fuse more GNN patterns on different ApplyEdge and ApplyVertex,automatically?
  • How to implement fused Aggregate
    GNN 101

    文章插圖
    Challenge:
    • Different graph data structureslead to different implementations in same logic operations;
    • Different graph characteristics favors different data structures;(like low-degree graphs favor COO, high-degree graphs favor CSR)
    • How to find the applicable zone for each and hide such complexity to data scientists?
More
  • Inference challenge
    • GNN inference needs full batch inference, how to make it efficient?
    • Distributed inference for big graph?
    • Vector quantization for node and edge features?
    • GNN distilled to MLP?
  • SW-HW co-design challenge
    • How to relief irregular memory access in scatter-gather?
    • Do we need some data flow engine for acceleration?
  • …
Finishing words“There is plenty of room at the top” 對技術人員很重要 。但為避免入寶山而空返，我們更需要建立起技術架構，這就像是地圖一樣，只有按圖索驥才能更好地探索和利用好top里的plenty of room 。
GNN 101

文章插圖
References
  1. Graph + AI: What’s Next? Progress in Democratizing Graph for All
  2. Recent Advances in Efficient and Scalable Graph Neural Networks
  3. Crossing the Chasm – Technology adoption lifecycle
  4. Understanding and Bridging the Gaps in Current GNN Performance Optimizations
  5. Automatic Generation of High-Performance Inference Kernels for Graph Neural Networks on Multi-Core Systems
  6. Understanding GNN Computational Graph: A Coordinated Computation, IO, And Memory Perspective

    推薦閱讀