PROVEE - PROgressiVe Explainable Embeddings

Deep Neural Networks (DNNs), and their resulting latent or embedding data spaces, are key to analyzing big data in various domains such as vision, speech recognition, and natural language processing (NLP). However, embedding spaces are high-dimensional and abstract, thus not directly understandable. We aim to develop a software framework to visually explore and explain how embeddings relate to the actual data fed to the DNN. This enables both DNN developers and end-users to understand the currently black-box working of DNNs, leading to better-engineered networks, and explainable, transparent DNN systems whose behavior can be trusted by their end-users.

Our central aim is to open DNN black-boxes, making complex data understandable for data science novices, and raising trust/transparency are core topics in VA and NLP research. PROVEE will advertise and apply VA in a wider scope with impact across sciences (medicine, engineering, biology, physics) where researchers use big data and deep learning.

📝 Table of Contents

• About
• Getting Started
• Feature/Performance Comparison
• Deployment
• Usage
• Built Using
• TODO
• Contributing
• Authors
• Acknowledgments

🧐 About

In this repository you will find PROVEE, short for Progressive Explainable Embeddings, a visual-interactive system for representing the embedding data spaces in a user-friendly 2D projection. The idea behind Progressive Analytics, such as described e.g. by Fekete and Primet, is to provide a rapid data exploration pipeline with a feedback loop from the system to the analyst with a latency below about 10 seconds. Research has shown that when performing exploratory analysis humans need a latency below about 10 seconds to remain focused and use their short-term memory efficiently. Therefore, PROVEE's goals are (1) to provide increasingly meaningful partial results as the algorithms execute, (2) provide visualizations that minimize distractions by not changing views excessively, (3) will provide cues to indicate where new results have been found by analytics, (4) should provide an interface for users to specify where analytics should focus, as well as the portions of the problem space that should be ignored. Note that these goals are adapted from the aforementioned publication.

PROVEE's architecture includes (1) back-end analysis algorithms (particularly, incremental projection algoritms), (2) intuitive, web-based user interfaces/visualizations and (3) intermediate data storage and transfer. Core to our system is an innovative, progressive analysis workflow targeting a human-algorithm feedback-loop with a latency under ~10 seconds to maintain the user's efficiency during exploration tasks. PROVEE will be scalable to big data; generic (handle data from many application domains); and easy to use (requires no specialist programming from the user). Please also refer to our Performance and feature comparison to available (visualization and analysis) tools

🏁 Getting Started

These instructions will get you a copy of the project up and running on your local machine for development and testing purposes. See deployment for notes on how to deploy the project on a live system.

Prerequisites

Docker, Kubernetes

Installing

To install the backend please refer to the backend README and for the frontend to the frontend README.

Generate gRPC service classes whenever protos change.

clone the latest Provee directory from Gitlab

git clone https://git.science.uu.nl/vig/provee/dummy-vue-grpc

cd to the cloned provee backend directory

cd dummy-vue-grpc/backend

run docker-compose to start all backend services

docker-compose up

cd to the cloned provee frontend directory

cd dummy-vue-grpc/frondent

ensure Node.js and yarn are installed

build the frontend

yarn install
yarn build

🔧 Running the tests

Explain how to run the automated tests for this system.

Break down into end to end tests

Explain what these tests test and why

Give an example

Basic Unit Tests

We use Jest for JavaScript based Unit Tests,

npm run tests

🎈 Usage

Notes about how to use the system are TBD, Video coming soon.

🚀 Deployment

If you want to deploy a live system refer to the Deployment Guide.

⛏️ Built Using

• VueJs - Web Framework
• NodeJs - Server Environment
• PixiJS - Visualization
• D3js - Visualization

✍️ Authors

• Michael Behrisch - Idea & Initial work
• Simen van Herpt - Backend & Infrastructure
• Jan Zak - Vis coding

See also the list of contributors who participated in this project.

🎉 Acknowledgements

• Hat tip to anyone whose code was used