Kubernetes

Deploy with Minicube

In order to run Emcee you have to configure and launch Kubernetes resources.

Minikube will be in use to deploy Kubernetes cluster. Or you can use any other predeployed Kubernetes cluster.

After installing Minikube run command to start a cluster:

minikube start

Configuration Kubernetes resources

We will need Deployment for workers, queue server and Artifactory as well as Service for queue and Artifactory.

Configuration file emcee-worker-deployment.yaml has Deployment for allocation of 3 workers.

Pay attention that env defines environment variables needed for worker itself. Learn more in worker configuration.

emcee-worker-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: emcee-worker-deployment
  labels:
    app: emcee-app
spec:
  replicas: 3
  selector:
    matchLabels:
      component: emcee-worker
  template:
    metadata:
      labels:
        component: emcee-worker
    spec:
      containers:
        - name: emcee-worker
          image: avitotech/emcee-worker:latest
          imagePullPolicy: IfNotPresent
          resources:
            requests:
              cpu: "2"
              memory: "4.5Gi"
            limits:
              cpu: "2.2"
              memory: "6Gi"
          volumeMounts:
            - mountPath: /dev/kvm
              name: kvm
          securityContext:
            privileged: true
          ports:
            - containerPort: 41001
          args: [ "$(EMCEE_WORKER_LOG_LEVEL)" ]
          env:
            - name: EMCEE_WORKER_QUEUE_URL
              value: "http://emcee-queue-service:41000"
            - name: EMCEE_WORKER_LOG_LEVEL
              value: "info"
      volumes:
        - name: kvm
          hostPath:
            path: /dev/kvm

Configuration file emcee-queue-deployment.yaml has Deployment for a queue server:

emcee-queue-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: emcee-queue-deployment
  labels:
    app: emcee-app
spec:
  selector:
    matchLabels:
      component: emcee-queue
  template:
    metadata:
      labels:
        component: emcee-queue
    spec:
      containers:
        - name: emcee-queue
          image: avitotech/emcee-queue:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 41000

Configuration file emcee-queue-service.yaml has Service for a queue server:

emcee-queue-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: emcee-queue-service
spec:
  type: NodePort
  selector:
    component: emcee-queue
  ports:
    - name: emcee-queue-service-ports
      protocol: TCP
      port: 41000
      targetPort: 41000

Configuration file emcee-artifactory-deployment.yaml has Deployment for Artifactory:

emcee-artifactory-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: emcee-artifactory-deployment
  labels:
    app: emcee-app
spec:
  selector:
    matchLabels:
      component: emcee-artifactory
  template:
    metadata:
      labels:
        component: emcee-artifactory
    spec:
      containers:
        - name: emcee-artifactory
          image: docker.bintray.io/jfrog/artifactory-oss:7.63.11
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8081
            - containerPort: 8082
          volumeMounts:
            - mountPath: /var/opt/jfrog/artifactory
              name: artifactory
      volumes:
        - name: artifactory
          emptyDir: {}

Configuration file emcee-artifactory-service.yaml has Service for Artifactory:

emcee-artifactory-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: emcee-artifactory-service
spec:
  type: NodePort
  selector:
    component: emcee-artifactory
  ports:
    - name: emcee-artifactory-service-port-1
      protocol: TCP
      port: 8081
      targetPort: 8081
    - name: emcee-artifactory-service-port-2
      protocol: TCP
      port: 8082
      targetPort: 8082

Launching Kubernetes resources

Set of resources should be launched in row:

kubectl apply --filename ./emcee-queue-deployment.yaml

kubectl apply --filename ./emcee-queue-service.yaml

kubectl apply --filename ./emcee-worker-deployment.yaml

kubectl apply --filename ./emcee-artifactory-deployment.yaml

kubectl apply --filename ./emcee-artifactory-service.yaml

Or using just single command. To do it put all configuration files in one directory and run:

kubectl apply --filename ./

To check the status of deployment run:

kubectl get pods

Output will look like:

NAME                                           READY   STATUS    RESTARTS   AGE
emcee-artifactory-deployment-db979bd5b-7fdbm   1/1     Running   0          6m5s
emcee-queue-deployment-5b49cf5884-wkmv9        1/1     Running   0          6m5s
emcee-worker-deployment-66c6b8bdb4-2kxqq       1/1     Running   0          6m5s
emcee-worker-deployment-66c6b8bdb4-96rfs       1/1     Running   0          6m5s
emcee-worker-deployment-66c6b8bdb4-pbtjv       1/1     Running   0          6m5s

Obtaining network address of queue and Artifactory

To get queue's local address run:

minikube service emcee-queue-service --url

The output contains ip:port of queue service which may be used in Gradle plugin, emceeplan or Nomad config.

To get Artifactory's local address run:

minikube service emcee-artifactory-service --url

The output contains ip:port of Artifactory which may be used in Gradle plugin, emceeplan as artifactory address.

If Emcee is deplyed on remote Kubernetes cluster then access to it should be aligned with your network configuration and Kubernetes settings.