Proto-OKN Theme 2: SPIDER: Scalable Public Infrastructure for Distributed Entity Relationships

The Scalable Public Infrastructure for Distributed Entity Relationships, SPIDER, is an NSF Proto-Open Knowledge Network (Proto-OKN) Fabric project that is developing processes and a technical solution for interconnecting distributed information represented in the form of knowledge graphs. By enabling automated interconnection among such graphs from different sources and fields, new insights can be rapidly obtained that would be impossible within the silo of a single domain or dataset. SPIDER will benefit society by facilitating greater access to reliable knowledge for the public and empowering researchers to discover new knowledge and obtain new insights from information across many different disciplines. SPIDER also accelerates the development of data-driven artificial intelligence (AI) models in the quest to address complex societal challenges. The data-driven analysis processes and overall technical solution provided by SPIDER are designed to leverage vast and rich existing data sources in the quest for solutions to some of society’s most pressing and complex challenges. By providing access to AI methods and deploying state-of-the-art interfaces for extensive querying of data, SPIDER facilitates integrated use of vast amounts of available data. SPIDER employs a scalable federated infrastructure for distributed knowledge graphs, enabling powerful queries, automated interconnection among data elements, training of artificial intelligence models, tracking of provenance, and computation of confidence scores. The SPIDER processes and architecture incorporate the capabilities needed by other Proto-OKN projects. The distributed knowledge graph fabric implemented by SPIDER is agnostic to graph formats, provides robust automated entity resolution and interconnection, allows subgraphs to be stored and processed in distributed fashion, presents powerful provenance tracking, enables GPU-accelerated queries to handle large subgraphs, and supports AI model training at scale. The solution also incorporates methods for utilizing trust networks to yield confidence scores on results, enabling participants to join the network with minimal barrier without the risk of polluting results with misinformation and without requiring a central oracle of truth. Furthermore, the solution allows for private data to be utilized in responding to queries while provably not leaking data. This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.