README.md 5.2 KB
Newer Older
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
1
# PSEUDo: Pattern Search, Exploration and Understanding in multivariate time series Data
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
2

3
[![](http://img.youtube.com/vi/oJfXoDyZRPY/0.jpg)](http://www.youtube.com/watch?v=oJfXoDyZRPY "")
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
4

Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
5
## Introduction
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
6
**PSEUDo** is an application that provides a way to search, explore and find interesting patterns in large multivariate time series data whilst giving a clear understanding of what's going on behind the scenes on the machine learning side. This is done by combining the vast knowledge of domain-experts with the immense processing power of computers, creating the interactive machine learning tool called **PSEUDo**.
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
7

Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
8
The application consists of two parts:
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
9 10 11 12 13 14
1. A client side, which is an Angular application
2. A server side, which is a Flask REST API written in Python

## Setup
To run the application, make sure you've installed the following:
1. Python >=3.5: https://www.python.org/downloads/
15
2. Conda: https://docs.conda.io/projects/conda/en/latest/user-guide/install/linux.html
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
16
3. Angular: https://angular.io/guide/setup-local
Yuncong Yu's avatar
Yuncong Yu committed
17 18
4. Java: https://www.java.com/en/download/help/download_options.html
5. `make` in git Bash in Windows: https://gist.github.com/evanwill/0207876c3243bbb6863e65ec5dc3f058 
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
19

20 21 22 23 24
### Step 1: Creating an environment
All dependencies are listed in the *environment.yml* file. To create an environment, run the following command:
`conda env create -f environment.yml`
This will create a conda environment named *pseudo*. Now activate the environment as follows:
`conda activate pseudo`
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
25

Yuncong Yu's avatar
Yuncong Yu committed
26
### Step 2: Prepare backend - creating the LSH package
27
The LSH algorithm is maintained locally for now, so you'll have to create it manually. The file that you need to setup this package is located in the Flaskserver folder (this is more efficient when debugging, as for every change you have to rebuild the package). So the package can be created by running the following code:
Yuncong Yu's avatar
Yuncong Yu committed
28
`cd Flaskserver\libs`
29 30 31 32
`python3 setup.py build_ext --inplace && python3 setup.py install`
`cd ..`
**NOTE 1**: So as a reminder, don't forget to run the 2nd line everytime you change something in the c++ code.

Yuncong Yu's avatar
Yuncong Yu committed
33 34
### Step 3: Prepare frontend - install Node packages
The cloned Angular repository cannot be used directly. You have to install the node packages via
Yuncong Yu's avatar
Yuncong Yu committed
35 36
`cd AngularApp/prototypes`
`npm install`
Yuncong Yu's avatar
Yuncong Yu committed
37 38

### Step 4: Running PSEUDo
39 40 41 42 43 44
As mentioned before, PSEUDo exists of a gui and a server. A Makefile is provided to setup both easily. Just run the following code for the server and gui respectively:
`make server`
`make gui`
A browser window should automatically be opened when running the GUI. If not, visit http://localhost:4200/
**NOTE 1**: Both gui and server will need their own terminal window.
**NOTE 2**: Make sure the pseudo environment is activated (step 1) when running the server, otherwise you'll get ModuleNotFound errors.
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64

# Documentation
## GUI
The GUI is an Angular framework. It consists of views (components) and a state (service). Every time an API call is done, the state will alter. Using hooks (EventEmitters), views can subscribe state variable changes and alter their view using the new value(s). The types of views is listed below.
### Overview
This view shows the entire dataset. When windows are labeled and candidates are generated, it should show the locations in this overview.
### Query
This view shows the current query.
### Training
This view shows the sampled candidates. In this view the user can label the samples as correct or incorrect. When the user is satisfied the labels will be updated and new samples will be generated.
### Progress
This view will show the progress of the learned classifier. It should show whether the classifier is getting better at understanding the desired pattern
### Labels
This view shows the currently labeled windows. The user should be able to change and delete labels in this view.

## API
### /read-data
This API call reads time series data from a file and returns the values and indices.
### /create-windows
This API call reads time series data and chunks it into windows. For now the windows are saved to a file locally.
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
65 66
### /get-lsh-parameters
This API call calculates the necessary LSH parameters based on the dataset
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
67 68
### /initialize
This API call starts the first iteration of the LSH algorithm. First some essential parameters are calculated. Then the LSH algorithm is called. The API returns the candidates, distances and hash functions.
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
69 70
### /weights
This API calculates the new weights for the hash function distributions, based on the relevance feedback given by the user.
Kruyff,D.L.W. (Dylan)'s avatar
Kruyff,D.L.W. (Dylan) committed
71 72 73 74 75 76 77 78
### /update
This API call runs the LSH algorithm with weights that will manipulate the hashing functions. The API returns the candidates, distances and hash functions.
### /query
This API will return the query data based on the provided window indices. If only one index is given, the API call will return the window values according to the index. If multiple indices are given, the DTW-based average of the window values according to the indices is returned.
### /window
This API simply returns the window values according to the index.
### /table-info
This API call will return extra info needed for the progression view. The input will be a subdivision of windows into buckets, and for each bucket the prototype (average) will be calculated + the distances between each prototype.