Note: < 5 minutes read
When running a virtualization workload on oVirt, a VM disk is 'natively' a disk somewhere on your network-storage.
Entering containers world, on Kubernetes(k8s) or OpenShift, there are many options specifically because the workload can be totally stateless, i.e they are stored on a host supplied disk and can be removed when the container is terminated. The more interesting case is stateful workloads i.e apps that persist data (think DBs, web servers/services, etc). k8s/OpenShift designed an API to dynamically provision the container storage (volume in k8s terminology).
See the resources section for more details.
In this post I want to cover how oVirt can provide volumes for containers running on k8s/OpenShift cluster.
Consider this: you want to deploy wikimedia as a container, with all its content served from
For that you will create a persistent volume for the container - when we have state to keep and server
creating a volume makes sense. It is persistent, it exists regardless the container state,
and you can choose which directory exactly you serve that volume, and that is the most important
part, k8s/OpenShift gives you an API to determine who will provide the volume that you need.
There are many options, Cinder, AWS, NFS and more. And in case the node that your pod is running on is a VM in oVirt, you can use ovirt-flexdriver to attach an oVirt disk and that will appear as a device in the node, and will be mounted with filesystem to your request. If you want to know more see the documentation about kubernetes-incubator/external-storage
k8s/OpenShift Node +-------> oVirt Vm +----------------------+ | | +----------------+ | mediawiki pod | | | | +---------------+ | | | | | | | | | | | | | | oVirt | | | | | | | | |/srv/mediawiki | | | | | +---------------+ | | | | | +----------------+ | | | | | /dev/pv001 (/srv/mediawiki) +-------> oVirt Disk | | +----------------------+
Checkout this youtube video, that demonstrate how it looks like in oVirt admin UI, kubernetes UI in cockpit, and some cli:
External Storage Provisioner and Flexvolume driver
OpenShift is able to request oVirt these special volumes by deploying ovirt-flexdriver and ovirt-provisioner and following this steps:
- Create a storage class
- Create a storage claim
- Create a pod with a volume that refernce the storage claim
- Run the pod
A storage class will can describe slow or fast data storage that maps to data domains in oVirt
kind: StorageClass apiVersion: storage.k8s.io/v1 metadata: name: ovirt-ssd-domain provisioner: external/ovirt parameters: type: io1 iopsPerGB: "10" ovirtStorageDomain: "prod-ssd-domain" fsType: ext4 ovirtDiskFormat: "cow"
When you create a storage claim, ovirt-provisioner will create an oVirt disk for you on the specified domain - notice the reference to the storage class:
# storage claim kind: PersistentVolumeClaim apiVersion: v1 metadata: name: mediawiki-data-ssd-disk annotations: volume.beta.kubernetes.io/storage-class: ovirt-ssd-domain spec: storageClassName: ovirt-ssd-domain accessModes: - ReadWriteOnce resources: requests: storage: 1Gi
Once the claim is created, oVirt is creating a 1Gb disk which is not attached to any node yet.
Run a mediawiki pod with so-called flex volume:
apiVersion: v1 kind: Pod metadata: name: mediawiki labels: app: ovirt spec: containers: - image: mediawiki name: mediawiki volumeMounts: - name: mediawiki-storage mountPath: "/data/" volumes: - name: mediawiki-storage persistentVolumeClaim: claimName: mediawiki-data-ssd-disk
And now it is the flexvolume driver job to tell oVirt to attach the disk into the node this
pod is running on, and creat file system on it, as described in the storage class, and to mount
it onto the node. When this is done, the volume is ready and the container can start, with
the mount set into the
/data directory as set by
Want to give it a try? Want to get updated about this?
This work as for today (Feb 20th 2018) is in progress and all of it can be found at the ovirt-flexdriver project page To deploy ovirt-flexdriver and ovirt-provisioner I created a container with Ansible playbook that takes an inventory that has the k8s nodes and k8s master specified, along with the ovirt-engine connection details. The playbook will copy and configure both component and get you up and running with just few keystrokes. Find more on deployment in the README.md of project and in an up-coming short video demonstrating the deployment.
Note on versions: this should be working against kubernetes 1.9 and oVirt 4.2 but 4.1 should work as well (because the API in use is the same).