[![VizierDB](data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAACAAAAAgCAYAAABzenr0AAAABmJLR0QA/wD/AP+gvaeTAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAAB3RJTUUH4QkPFAYcHcvgxQAAABl0RVh0Q29tbWVudABDcmVhdGVkIHdpdGggR0lNUFeBDhcAAAa/SURBVFjDxZfNcxxHGcZ/3T0f+6FvyZK9lstZ+UuyHFTBH5iDSQWKCpThkuToEweKK38AVRyponINUMUBnGPCAVypYEgIxCZxbMXYjqzYlmRJlleOFMmSVrszOzvT3RxmtZbKqeCPIpmtrd2umZ5++nnf93mfFq/9/KTla7wkX/PlbB6IL3jAflUAhBA4no8UEttYViBI4gidJCDE/5kBAVIqpJTYJhsCkcRfXQjAYm3zH8Yk6CRG6ySFIxVSqkZgBNZqjNYP9iAlUjk0X/L4AFKmjdZYIJNvx8/kieM6UirqYYV6FCKVg9ExyvXJtXWBtQghiaOAWnUd5XpPAkAAAmM0SMXgsRcZPPI8cRRiGwuAYfTt09y8fI6+gRFOvPRTsvk8OklQyiWurXP1vTeYvHYJN5N9JCaaAKwxacLFEW6uk/7BY3QXig9N2LnnIGMfnqV9e5Hdg889dH+ldIsbH5/Hy2QfqYLUyW/v/+VmEABJvUYtWKd7R5FsSztCQD2sMH31X4x/9A7B+hpGx0gnQ0tHN14mC8Dk6DtcO3eGMKg2cuURhchai5SSfGs7bV29dHRt4/Opy3zw598TVNYAiKqrXPrraUq3b5Bt6aCyNMdHb/2BlaVFABZv/4ezr/+a0swEjuc/vhIKKclkcmRzeTK5PK2d26gszbGyUAKgtaefXYOH0XGEkIIkSdjWv5f+PYMATF05R70W4PqZx6oCuVnyrDVYk36FkAibMPvpZUzjfTv3jpBryRNHNRw/z87BIwjAxgF3J8cBgVLO0/SCrWonHY+ZsX8TVsoAbC8O0du/h6C8Qr69m+Kh4wDcvTnK2vJnKNd/imYkQEjRVEMpJY7jUr0/z8yNqwBk27ezvTiENZru/gNs274DgNufXKCydh/nMep/CwDRqIAoDAiDCrWgSi2oEtUCknrE1JX3mxMKA8O09/Sxa+gYAOHaPRbuTIJ0EU/QL5wN+dNas7qyhDX2QSQs6LjG0p0brN5fpqOrm8K+EfqeGWZncT8ApYlrLJWm8TK5J+qdzkYZKsehs6cXx/Ww1qS6KCSrywsE5WWmrn7I4Rd+hN/Sw9EfnKKrN6V/fnKMWq1Gvi2XquiT5YBFSUVbRxdd3b10dvXS0d1LZ08fmVwrUbDO3ZujzUn9+0dw/QzlxVnuTlzB9XNYa57OEVks1hiMMRijMcZgjQYEruexOHuD+ZmpLZMXZq6zMDfdYM3+D2vzpb1AbHI/G/3YNseO5xOUlxg7/xd8/yWwhqi6yviFvyNdf0vsrTXN5iWEwFrbGIsvTFJnY/8CgRQi7fkb9EgJQmCNxXU9Ji79jbnxC7ieT70WEFbLeH520+IWqRyUVGgdY0y6sFIKY8yXM2CMIQwqJEmcVkJDnpO4npoEIRBAWFklsKlSbq77KKzSsa3Ad1/5CW1tLUyPXeTj8//kGyd+yNDwIc6//SbXL18kk8ttUUu54Qe1Tvj8Xon52dvcm5vm3tw087NThNUKUspGXmgQEikdECloIQRGJyRJQhLXmfhkFOn4FA8cJKmHrC4v0frMMK1t7egkSfWmFjZzZksZtrZ14rhu86YUkvLqMvV6nb7CbvxMFmNt0y+ul1dYXV5ACMXQ0e9w8JvHWbp3h/LqMlnHkNQjSlPjxPcX2bl3mJOFAaRNGLv4PgulORzXewDAcVwKuwfItbQ3fZ5yHCauXyZeWWbXniE6evowSYKQaYLNTIzx2Z0pnnvhx3zv5VMszd7C9fbRv3+Y+fGLCClRSqGR7D4wzNzULboLRYqDh3jr9GvMl+a2NiOjDUZrtE5SyrVutlajNTpJ0LphVJP018218ey3nideW+TN3/6KN373KqXJW/i+n3pLY/F9j2vnz/L6q7/gzOnfkC3sYejwcZKothmAbX42WHkw4kFp2tQ528ZYOR6O6xBHNZJYY7ShXgsRUmJsqilCCqIwxBhDLagACtf3wdqtplQqhVIOohFlpZxm7UqlUK6bVoMQKb2uR1heZmrsCke/f5KTp35GUKtTPPgsJiyze98QnYV9kM0xcuJF/NYudgwMY1ZK3L5+FeX6KQDREJCwWgFjG5qeAtKNg0m1vJruSuumqNSCAM/3GP3HGYxJGBw5glq8y9gH79Le1UtLWwd+Nsf0pfeoBSGFgSGEiXj3T39k4vo1vEwG0TwdC4GS6iG10jppJqmQcpPdEmiTYLTG6IQ4jlNajaEeRSDA9TJYo0niGCEEfiab5pDWuJ6fHgc3xzdJ4q0t1aZiBBDH0UNWb4MJqRx85aSuWioyufwm362agmWtSZXScZu3/wvzV1IhqXS0oAAAAABJRU5ErkJggg==)](https://vizierdb.info) # Deploy a Containerized Vizier Instance Vizier is a cloud-enabled tool that makes it easy to explore, validate, transform and debug data. - vizier core platform is open-source. See: [api](https://github.com/vizierdb/web-api), [ui](https://github.com/vizierdb/web-ui), [mimir](https://github.com/UBOdin/mimir) on GitHub - for more details about Vizier see: ### Components Vizier has a number of components that are not trivial to set up manually: * [Web UI] - React user interface! * [API Server] - python wsgi api server * [MimirDB] - virtual probabalistic database * [Proxy] - a reverse proxy that provides an endpoint for vizier services * [Apache Spark] - distributed data processing * [Hadoop] - distributed data processing * [S3] - optional data staging endpoint * [Analytics] - optional vizier ui access tracking Though instalation instructions for each of these components is availabel, it is time-consuming and difficult to install them manually. So is there an easier containerized deploment? Yes! Deployments to a kubernetes cluster and to docker are explained below. ### Deploy The Vizier Stack to docker If you already have a docker installed, good. If not, you can get installed pretty fast. See [docker](https://docs.docker.com/install/) for more details, but on Ubuntu, basically you can just do the following: ```sh sudo add-apt-repository \ "deb [arch=amd64] https://download.docker.com/linux/ubuntu \ $(lsb_release -cs) \ stable" sudo apt-get update sudo apt-get install docker-ce docker-ce-cli containerd.io ``` Once your docker instalation is ready, get the [bash script](https://vizierdb.info/assets/run-vizier-containers.sh) for deploying Vizier and make the following adjustments: - update the VIZIER_DOMAIN variable for the vizier-proxy deployment to the domain you will use to access Vizier. You can use a real domain and DNS entries or the hosts file of a client. ([run-vizier-containers.sh](https://vizierdb.info/assets/run-vizier-containers.sh): Line 26) - update the name or host paths for the volumes if you would like them somewhere other than the default ([run-vizier-containers.sh](https://vizierdb.info/assets/run-vizier-containers.sh): Line 31) - update the s3-credentials and bucket name with your S3 access key id, secret, and bucket name: ([run-vizier-containers.sh](https://vizierdb.info/assets/run-vizier-containers.sh): Line 28, 29, 30) Deploy vizier ```sh ./run-vizier-containers.sh ``` The IP address of the vizier-proxy service for a local docker deployment will likely be 127.0.0.1 ### Deploy The Vizier Stack to Kubernetes If you already have a [kubernetes](https://kubernetes.io/) cluster set up, good, make sure CoreDNS is enabled (we are using k8s v1.13.2). If not, you can get a single node cluster setup pretty fast using [microk8s](https://microk8s.io/). See [microk8s docs](https://microk8s.io/docs/) for more details, but basically you can just do the following: ```sh sudo snap install microk8s --classic microk8s.enable dns dashboard ``` Once your cluster is ready, get the [yaml file](https://vizierdb.info/assets/vizier-deployment.yaml) for deploying Vizier and make the following adjustments: - update the host paths for the persistent volumes if you would like them somewhere other than /mnt/ (YAML Line 15, 28, 41, 310) - update the s3-credentials secret with your S3 access key id and secret - base64 encode them first: (YAML Line 330, 331) ```sh echo "YOUR-S3-ACCESS-KEY-ID" | base64 echo "YOUR-S3-ACCESS-KEY-SECRET" | base64 ``` - update the VIZIER_DOMAIN env variable for the vizier-proxy deployment to the domain you will use to access Vizier. You can use a real domain and DNS entries or the hosts file of a client. (YAML Line 622) Deploy vizier ```sh kubectl create -f vizier-deployment.yaml ``` You may need to do this to allow containers to access the internet ```sh sudo iptables -P FORWARD ACCEPT ``` Find the ClusterIP or ExternalIP of the vizier-proxy service ```sh kubectl get service vizier-proxy ``` ### After Deployment After you have the IP of the vizier-proxy service you need to add the following entries to either DNS for a real domain or the hosts file of the client: so where VIZIER_DOMAIN=vizier.dev | IP Address | Host Name | Purpose | | ------ | ------ | ------ | | IP of vizier-proxy service | demo.vizier.dev | web ui for vizier | | IP of vizier-proxy service | api.vizier.dev | web api for vizier | | IP of vizier-proxy service | vizier.vizier.dev | supervisor ctl for api | | IP of vizier-proxy service | mimir.vizier.dev | supervisor ctl for mimir | | IP of vizier-proxy service | proxy.vizier.dev | supervisor ctl for proxy | | IP of vizier-proxy service | analytics.vizier.dev | endpoint for access analytics and ui | | IP of vizier-proxy service | spark.vizier.dev | web ui for spark master | | IP of vizier-proxy service | driver.vizier.dev | web ui for spark driver | | IP of vizier-proxy service | hdfs.vizier.dev | web ui for hadoop | Now you should be able to access the Vizier UI from a web browser. ``` https://demo./vizier-db ``` License ---- Apache License 2.0