Installation

Here is the detailed process of installation of ApertureData Platform dependencies.

Dependencies

sudo apt install scons
sudo apt install libjsoncpp-dev
sudo apt install automake libtool cmake make g++
sudo apt install curl wget unzip
sudo apt install libgtest-dev libgoogle-glog-dev
sudo apt install zlib1g-dev libbz2-dev libssl-dev liblz4-dev
sudo apt install libtiff5-dev libjasper-dev libgtk-3-dev
sudo apt install ffmpeg

// Also, install one of the following for MPI
sudo apt install libopenmpi-dev
sudo apt install mpich

Install gtest

Unfortunately apt doesn’t build gtest; you need to do the following steps to get it to work correctly:

cd /usr/src/gtest/
sudo cmake CMakeLists.txt
sudo make -j
sudo cp *.a /usr/lib

Protobuf

Google Protobufs. Default install location is /usr/local.
git clone https://github.com/google/protobuf.git
cd protobuf && git checkout v3.9.0 && ./autogen.sh
./configure --prefix=/usr/local
make -j
sudo make install

OpenCV

Below are instructions for installing OpenCV v3.3.1. It may also work with newer versions of OpenCV. Download OpenCV 3.3.1.

wget -O opencv.zip https://github.com/opencv/opencv/archive/3.3.0.zip
unzip opencv.zip && rm opencv.zip && cd opencv-3.3.0
mkdir build && cd build
cmake -D CMAKE_BUILD_TYPE=Release \
-D CMAKE_INSTALL_PREFIX=/usr/local ..
make -j
sudo make install

Note: When using videos, and getting the following error: “Unable to stop the stream: Inappropriate ioctl for device”, you may need to include more flags when compiling OpenCV. Follow these instructions ([source](https://stackoverflow.com/questions/41200201/opencv-unable-to-stop-the-stream-inappropriate-ioctl-for-device)):

sudo apt install ffmpeg
sudo apt install libavcodec-dev libavformat-dev libavdevice-dev

// Rebuild OpenCV with the following commands:

cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local  -D WITH_FFMPEG=ON -D WITH_TBB=ON -D WITH_GTK=ON -D WITH_V4L=ON -D WITH_OPENGL=ON -D WITH_CUBLAS=ON -DWITH_QT=OFF -DCUDA_NVCC_FLAGS="-D_FORCE_INLINES" ..
make -j7
sudo make install

TileDB

ApertureData Platform works with TileDB v1.3.1. The directions below will help you install TileDB v1.3.1 from source. You can also follow the directions listed here.

wget https://github.com/TileDB-Inc/TileDB/archive/1.3.1.tar.gz
tar xf 1.3.1.tar.gz && rm 1.3.1.tar.gz
cd TileDB-1.3.1 && mkdir build && cd build
../bootstrap --prefix=/usr/local/
make -j
sudo make install-tiledb

Faiss

Facebook Faiss library for similarity search, used as alternitive engines. Download Faiss 1.4.0.

wget https://github.com/facebookresearch/faiss/archive/v1.4.0.tar.gz
tar xf v1.4.0.tar.gz && rm v1.4.0.tar.gz
cd faiss-1.4.0 && rm CMakeLists.txt
mkdir build && cd build && cmake ../
make -j
sudo mkdir /usr/local/include/faiss/
sudo cp -r /faiss-1.4.0/* /usr/local/include/faiss/
sudo mv /faiss-1.4.0/build/lib/libfaiss.so /usr/local/lib/

You may need to change some flags in the CMakeFile depending on configurations on your system.

Things that we have needed to change in CMakeLists.txt: * If your system doesn’t have a GPU, make sure that BUILD_WITH_GPU is OFF * You may need to add -msse4 to set(CMAKE_CXX_FLAGS * Change the add_library call to be SHARED instead of STATIC

You can use our makefile:

wget https://gist.githubusercontent.com/luisremis/758c71c6a86cb3e4fe2311c415e07547/raw/3c51b5ed7d3f16a4d684d9ea524602c9f1f8593a/CMakeLists.txt

This library does not offer an install. Make sure you move .h files to /usr/lib/include/faiss or /usr/local/lib/include/faiss, and make sure you make the library (libfaiss.so) is available system-wide Or follow instructions [here](https://github.com/facebookresearch/faiss/blob/v1.4.0/INSTALL.md)

Valijson

This is a headers-only library, no compilation/installation necessary

git clone https://github.com/tristanpenman/valijson.git
cd valijson
sudo cp -r include/* /usr/local/include/

Persistent Memory Graph Database (PMGD)

Python Client Module

ApertureData Platform offers the Python Client Module through the pip package manager, and it is compatible with Python 2.7+ and 3.3+. pip (or pip2 and pip3) will automatically install dependencies (protobuf).

pip install vdms

Compilation

git clone https://github.com/intellabs/vdms
// Or download a release.

cd vdms
scons [FLAGS]
Flag Description
–no-server Compiles client libraries (C++/Python) only. (will not compile neither server nor tests)
–timing Compiles server with chronos for internal timing, experimental.
-jX Compiles in parallel, using X cores
AD_PATH=path Path to the root folder containing pmgd. Default is “./” which is pmgd inside aperture data folder. Example: scons AD_PATH=/opt/ad/

Running The ApertureData Platform Server

The config.json file contains the configuration of the server. Some of the parameters include the TCP port that will be use for incoming connections, maximun number of simultaneous clients, and paths to the folders where data/metadata will be stored.

We provide a script (run_server.sh) that will create some default directories, corresponding the default values in the config-vdms.json.

To run the server using the default directories and port, simply run:

sh run_server.sh