Beaker's Hardware Inventory System

class: center, middle

# Beaker's Hardware Inventory System
      ### LCA 2015
      ##  Nick Coghlan, Amit Saha
      ### Red Hat Inc.

---

# Agenda
      
      0. What is Beaker?
      
      1. Beaker's hardware inventory

2. How is the inventory created?

3. Replacing smolt by lshw
      
      ---

# What is Beaker?

* Full stack integration testing system

* Supports running tests on Fedora, CentOS, Red Hat Enterprise Linux, Docker hosts such as Atomic images

* Run tests on baremetal/OpenStack VM/docker container

* A test can specify the hardware (architecture, memory, disk space, devices ..) and distro

* LCA 2014 [talk](https://www.youtube.com/watch?v=tjUjdBm-Mqw) by Nick Coghlan
        * ([Lot](https://beaker-project.org/docs/whats-new/) has happened since then!)

* Learn [more](https://beaker-project.org)

---
      
      # Beaker's hardware inventory

* Hardware database maintaining record of the hardware for each system in Beaker
      
      * Supported architectures, memory, CPU details, disk, hypervisor, devices, etc.

## Making use of the inventory data

* Job XML specification

* ```<family op="=" value="15"/><model_name op="like" value="%Intel%"/>```

* can get longer, difficult to type in/remember

* A friendlier way - *pre-defined host filters*

* ``bkr workflow-simple --host-filter INTEL__FAM15_CELERON ..``

---
      
      # How is the inventory created?

* Run a Beaker job with the [/distributon/inventory](https://git.beaker-project.org/cgit/beaker-core-tasks/tree/inventory) task
      
      * Makes use of [beaker-system-scan](https://git.beaker-project.org/cgit/beaker-system-scan/)
      
      * The system is provisioned, the task is run and the inventory data is sent to Beaker's database

```
      Data:
      {'Arch': ['ppc64'],
      'Cpu': {'CpuFlags': [],
      'cores': 0,
      'family': 0,
      'model': 4915712,
      'modelName': 'POWER7 (architected), altivec supported',
      'processors': 4,
      'sockets': 0,
      'speed': 4116.0,
      'stepping': 0,
      'vendor': 'IBM'},
      ..
         ```

---
      
      # beaker-system-scan

* Mostly Python, can be used outside Beaker (``-d`` for debug mode)

```
      # beaker-system-scan -d
       Data:
       {'Arch': ['ppc64'],
        ..
      
       'model': 4915712,
       'modelName': 'POWER7 (architected), altivec supported',
       'processors': 4,
       ..
       'Disk': {'Disks': [{'model': 'AIX VDASD',
       'phys_sector_size': 512,
       'sector_size': 512,
       'size': '429496729600'}]},
       'Numa': {'nodes': 2},
       'memory': 15780,
       ..
      
       ```

* Uses ``python-linux-procfs``, ``libparted``, and
      
        * The ``master`` branch uses [smolt](http://fedoraproject.org/wiki/Smolt)
        * The ``lshw`` branch uses [lshw](http://www.ezix.org/project/wiki/HardwareLiSter)
     
      ---
      
      # Replacing smolt by lshw

* ``smolt`` has been [retired](https://fedoraproject.org/wiki/Smolt_retirement) for a while

* ``beaker-system-scan`` doesn't work on RHEL/CentOS 7+ and Fedora

* Hence, for example, ``aarch64`` hardware cannot be inventoried

* [Plan](https://beaker-project.org/dev/proposals/inventory-lshw-migration.html)

* Use ``lshw`` instead in ``beaker-system-scan``

* ``lshw -xml``
      
        * Parse the XML data for the relevant information
      
      * ``lshw`` based task will support: ``ia64, i686, x86_64, s390x, ppc64, arm, aarch64``

---

# Replacing smolt by lshw

* Beaker project maintains a [lshw fork](https://git.beaker-project.org/cgit/lshw)
      
      * Maintaing a fork helps the Beaker team in trying out fixes and enhancements quickly

* Work with upstream maintainer to merge patches

* Enhanced lshw's functionality on ``ppc64, s390x, arm, aarch64`` hardware ([PR](https://github.com/lyonel/lshw/pull/2/commits))

---

# Replacing smolt by lshw: Comparison

* Are we missing any important information that ``smolt`` gave us (and ``lshw`` doesn't?)
      
      * Are we collecting the wrong information?

* Compare the data that ``smolt`` and ``lshw`` gives us on all supported architectures
      
          * ``arm`` and ``aarc64`` comparison not possible (no ``smolt`` supported distro)

* Sample results for a ``ppc64`` KVM guest is next

---

# Replacing smolt by lshw: Comparison

![Alt text](comparison_cpu.png "CPU data comparison")

---

# Replacing smolt by lshw: Comparison

![Alt text](comparison_devices.png "Devices comparison")

---

# Replacing smolt by lshw

* ``beaker-system-scan`` with ``lshw`` looks good overall

* ``lshw`` doesn't *seem* to do well w.r.t retrieving devices on non-x86 archs
      
      * Thorough comparison still remains to be done

* We expect some fixes will be required in both ``beaker-system-scan`` and ``lshw``

# Learn more
      
      * Design proposal: [Migration to lshw for inventory task](https://beaker-project.org/dev/proposals/inventory-lshw-migration.html)
      
      * [Beaker developers mailing list](https://lists.fedorahosted.org/mailman/listinfo/beaker-devel)

---

# People involved

* James de Vries (past intern) started off the replacement
      
      * Current Beaker development team
           
      # Thanks

* Nick Coghlan for presenting this talk
      
      * All the Beaker team members

* LCA 2015 team