Friday, February 1, 2019

ACPI Debugging (3) - Debug ACPI Tables with FWTS

Previous two articles, "ACPI AML Debugger in Ubuntu 18.04" & "Debug AML (DSD & SSDT) with ACPICA Utilities", discuss techniques for AML, debugging; however ACPI specification includes many other important tables than DSDT and SSDTs.

While other ACPI tables include fixed fields and lengths, it doesn't mean their contents cannot go wrong. In fact, many systems in the field have many incorrect ACPI tables in their firmware. This can happen when systems are developed by integrating code from various parties without modified for customized hardware.

A test suite is a good solution to catch errors without involving too many engineering resources as it can identifying errors automatically or semi-automatically. In particular, Firmware Test Suite (FWTS) is one of the best test suite (did I also mention it is freely available too?) and it is the recommended ACPI test suite by UEFI forum, who is the owner of ACPI specification.

What is Firmware Test Suite (FWTS) ?

Firmware Test Suite (FWTS) is a test suite that performs sanity checks on firmware. It is intended to identify BIOS, UEFI, ACPI and many other errors and if appropriate it will try to explain the errors and give advice to help workaround or fix firmware bugs.

More information about FWTS can be found at its official website or in a previous blog.

Installing FWTS

FWTS be can installed easily with package managers on many Linux distros. It is developed on Ubuntu Linux and can be installed as follows:
  1. Add stable PPA for latest FWTS (optional) - sudo add-apt-repository ppa:firmware-testing-team/ppa-fwts-stable
  2. Install FWTS - sudo apt install fwts
FWTS packages for Other Linux distros can be found here.

Using FWTS for ACPI Errors

Running FWTS


FWTS is a command-line tool and its usage is "fwts cmds" (root access is usually required). The following are examples of fwts checks ACPI tables

  • Check FADT - fwts facp
  • Check MCFG - fwts mcfg
  • Check table checksums- fwts checksum
  • Check table signatures and IDs - fwts acpitables

Below is an example of how fwts was executed.



More details of FWTS's ACPI tests can be found on its references website or by the command fwts --show-tests.

Running Multiple Tests


FWTS can also run multiple tests by appending tests one after another as below:



Running All ACPI Tests

FWTS also provides a way to execute all tests in a category. For example, sudo fwts --acpitests run all ACPI tests all together. A similar command is sudo fwts --uefitests (which will not be discussed in details here).

Let's See Some Examples...

FWTS is used for real projects. Here are some examples of tests we have run.

Checking FADT


Running sudo fwts fadt executes 6 tests that checks whether system firmware/BIOS implement FACP table correctly.

Especially, the sizes of FACP table grow with ACPI specs, but some firmware/BIOS updates the FACP major and minor versions without adding the new fields in later ACPI spec. This can be caught by FWTS with ease. An example is Test 3 in results.log shown below.


 

Checking Table Checksum


Another common error in system firmware/BIOS is ACPI table checksums. When built by AML compiler, ex. iasl, table checksums are updated accordingly; however, it is a common practice that system firmware/BIOS modifies tables for hardware customization during boot time and this can corrupt tables if firmware/BIOS does not update checksum fields accordingly.

Below is an example of running sudo fwts checksum to verify the checksum fields of all ACPI tables.


Next Topic...

FWTS can analyze ACPI tables from other systems. This can be done with fwts --dumpfile=acpidump.log. More details will be discussed in the next blog.

Thursday, January 10, 2019

ACPI Debugging (2) - Debug AML (DSDT & SSDT) with ACPICA Utilities

Using acpidbg on Ubuntu 18.04 x64 can be quite handy; however, the Linux kernel with ACPI_DEBUGGER is not always available, such as on Ubuntu for ARM. In such cases, acpica also provides a set of utilities, named acpica-tools, for ACPI debugging.

Installation

The acpica-tools can be installed by
  •  sudo apt install acpica-tool
The latest source code can be downloaded from either of below, and it can be compiled and installed by “make” followed by “sudo make install“.
  • Windows: https://acpica.org/downloads/binary-tools
  • Ubuntu: sudo apt install acpica-tool

ACPICA Tools

The acpica-tools consist of the following utilities: acpibin, acpiexamples, acpihelp, acpisrc, iasl, acpidump, acpiexec, acpinames and acpixtract. This article focuses on how to use iasl, acpidump, acpiexec and acpixtract because they are commonly used in debugging.
  • acpidump - collect ACPI tables from a running system
  • acpixtract - extract ACPI tables from an acpidump file
  • acpiexec - emulate ACPI tables from extracted acpi tables
  • iasl - compile & disassemble ACPI tables

A Case Study - Airplane Mode

The airplane-mode button on laptops is usually implemented by both system BIOS and an OS driver. More specifically, it requires an ACPI device to be “present” in DSDT (or SSDT) table and a corresponding device driver. The figure below demonstrates how to debug if the airplane mode button fails to work.



Each computer brand has its specific implementation(s) and corresponding device driver(s). Some examples are listed below. Similarly, other driver source code can be found in Linux kernel.
  • Acer - acer-wireless
  • ASUS - asus-wireless
  • Dell - dell-rbtn, intel-hid, intel-vbtn
  • HP - hp-wireless
  • Lenovo - idealpad-laptop, thinkpad_acpi
  • Xiaomi - hp-wireless

Let’s use a Dell system as an example but the same technique also applies for others. To understand whether the airplane mode driver is loaded, one can run



As expected, the intel-hid is used on this particular Dell system.

As seen above, the “intel-hid” is used on this particular Dell system.
Let’s assume we know intel-hid is supposed to be used on this Dell system but it is not loaded for now. We can debug BIOS ASL/AML code using ACPICA utilities following the below steps.

Find Out ACPI Device's (intel-hid's) Hardware ID


A quick online search shows intel-hid.c is for an ACPI device with _HID “INT33D5”.

Extract and Disassemble ACPI Tables

  • Get all ACPI tables: sudo acpidump > acpi.log
  • Extract DSDT and SSDT: acpixtract acpi.log
  • Disassemble tables: iasl -d *.dat

Check Whether INT33D5 is "Present"

  • Find INT33D5 in DSDT or SSDT: grep INT33DT *.dsl

Once found, let's use vi to see more details in DSDT such as the device name (HIDD) as below:


  • Check device status, i.e. Method (_STA)
In addition to the device declaration, Linux ACPI device driver is loaded when _STA returns “present”. As in the above ASL code, _STA can be present (return 0x0F) or absent (return Zero). One can continue to trace OIDE() -> OSID() and so on to find out what _STA returns.

Alternatively, using acpiexec is easier for this task.

Evaluate _SB.HIDD._STA with acpiexec


Using acpiexec is the same as acpidbg, but it loads ACPI tables from files and runs AML in emulation mode, i.e. it does not touch hardware registers or actual memory.

  • Load ACPI tables: acpiexec *.dat
  • Find HIDD path: find HIDD
  • Execute HIDD's _STA: execute _SB.HIDD._STA
If _SB.HIDD._STA returns 0xF but intel-hid is not loaded, something is wrong with Linux kernel, ex. intel-hid is not compiled or is blacklisted. In this case, this should be forwarded to a Linux kernel engineer.

If _SB.HIDD._STA returns 0, we can continue to use acpiexec to find out OIDE() returns. In fact, we can use acpiexec to trace _SB._HIDD._STA
  • Trace _SB.HIDD._STA: debug _SB.HIDD._STA
Tutorial of using acpiexec is the same as acpidbg and can be found in the previous blog.

Final Words

ACPICA tools are very useful for debugging ACPI bugs, and using it well can save much time. This tutorial touches surface of what ACPICA tools can do and more documents can be found on acpica.org website.

Tuesday, May 1, 2018

ACPI Debugging (1) - ACPI AML Runtime Debugger in Ubuntu 18.04 (x64)

ACPICA is an open-source project that provides an operating system (OS)-independent reference implementation. It also contains a list of utilities such as ASL compiler (iasl), acpiexec (an AML emulator) and so on. However, debugging AML on Linux in runtime wasn't provided in ACPICA ... not until Linux Kernel 4.13.

Enabling AML Debugging


The aml-debugger.txt, the instruction for enabling AML runtime debugger, is available at Documentation/acpi/ in Linux kernel source code. In short, two things are required for AML runtime debugging
  • Enable AML debugging (CONFIG_ACPI_DEBUGGER=y & CONFIG_ACPI_DEBUGGER_USER=m) when compiling Linux kernel
  • Compile an utility, acpidbg from Linux kernel (I uploaded a copy here)
While compiling a custom-build kernel is nothing new to kernel developers, it is often inconvenient for system firmware / BIOS developers who need to verify ACPI AML implementation in their firmware. Fortunately, Ubuntu 18.04 (x64) and later enable these config by default, and one can simply execute acpidbg on Ubuntu 18.04 - even on Ubuntu Live from USB too!

Running AML Debugging


Executing acpidbg on Ubuntu 18.04 (x64) is straight-forward


and "help" shows a list supported commands



Examples


acpidbg is particularly handy when one needs to evaluate any ACPI AML objects during runtime - especially ones that change under different conditions. This may be explained by some examples below:

Example 1 - To determine AC power status in runtime.
  1. Find _PSR objects
  2. Evaluate _PSR when AC is disconnected
  3. Evaluate _PSR when AC is connected.


acpidbg also works with complex objects such as packages.
Example 2 - To determine battery information and status.
  1. Find _BIF and _BST objects
  2. Evaluate _BIF and _BST objects (note _BST changes dynamically).


acpidbg can decode bit fields and save time on counting bits. Try to run acpidbg to evaluate any _PLD objects and you will see what I mean :).

Wednesday, November 22, 2017

[Presentation] Firmware Test Suite - Uses, Development, Contribution and GPL

I did a presentation of "Firmware Test Suite - Uses, Development, Contribution and GPL" in UEFI Plugfest in Taipei on Nov 1 2017. The presentation is shared @ slideshare and can be viewed below:

Monday, July 10, 2017

Changes of the _OSI object in ACPI spec 6.1a & 6.2 and in Linux

ACPI _OSI is the most controversial object in my opinions. Not only _OSI was not welcomed by Linux kernel but also it has been discussed multiple times in ACPI Spec Work Group (ASWG) meetings. Many spec contributers tried to remove _OSI from ACPI spec, but the impacts were too great to depreciate _OSI...

Until _OSI is now redefined in ACPI 6.1a & ACPI 6.2 as below



Still confused? Let's take a look at "Table 5-163 Predefined Operating System Vendor String Prefixes" in ACPI 6.1a below:

Operating System Vendor String Prefix Description
“FreeBSD” <FeatureGroupString> Free BSD OS features/interfaces
“HP-UX” <FeatureGroupString> HP Unix Operating Environment OS features/interfaces
“Linux” <FeatureGroupString> GNU/Linux Operating system OS features/interfaces
“OpenVMS” <FeatureGroupString> HP OpenVMS Operating Environment OS features/interfaces
“Windows” <FeatureGroupString> Microsoft Windows OS features/interfaces

While <FeatureGroupString> looks like something new, this addition is backward-compatible - it is very common to see "Windows 2009" (Windows 7), "Windows 2013" (Windows 8.1) and "Windows 2015" (Windows 10). 2009, 2013 and 2015 are sets of new features provided by each new Windows OS and is more than a name of each new OS. Does this make sense?

In fact, the main purpose of this _OSI change is to clarify "_OSI is not to be used to detect operating systems but feature sets supported by operating systems" - these aren't my words but the words of spec contributers in ASWG's meetings. In other words, please stop complaining why OSI("Linux") doesn't work - this usage is never the intention of ACPI spec.

While _OSI("Linux") does NOT  work in Linux kernel by default, kernel maintainers now agree Linux isn't always compatible with Windows and using _OSI("Windows xxxx") isn't the best idea. Each OEM can define different "Linux <FeatureGroupString>" in their ACPI BIOS, and OEM can submit patches so new strings will be recognized by Linux kernel.

The string should be in the format "Linux-OEM-my_interface_name" as defined here. I would guess a valid string can be something like "Linux-Dell-SuperVGA" or "Linux-HP-CrystalAudio" (I made them up, and they aren't recognized by Linux kernel). The kernel document discusses the history and the rules, and BIOS engineers should not abuse this mechanism - if you want to a workaround, create a new string for it, ex. "Linux-Dell-VGA-Workaround" and submit a new patch to Linux kernel. DO NOT use unrelated OEM strings for other purposes.

The new _OSI is introduced for very short time and no ACPI BIOS uses new Linux strings yet. It is our hope this can provide better compatibilities between ACPI BIOS and non-Windows OS's. If anyone has any feedbacks or concerns about this new _OSI thing, I am happy to answer or forward to ACPI Work Group for more discussion.

Friday, July 7, 2017

Why cannot ACPI _REV object be used to detect Linux anymore?

Linux community is never a fan of distinguishing Windows & Linux in BIOS (see my 2012's blog Why does Linux Say It Is not Linux in _OSI("Linux")?). However, many BIOS engineers seem to see the needs of to workaround BIOS bugs for Linux regardlessly. After realising _OSI("Linux") cannot be used by default, some creative BIOS engineers figured ACPI's _REV object could be used for this purpose some years ago, based on the following facts:

1. _REV definition in ACPI spec 5.1


2. Windows returns 2 & Linux returns ACPI revision


While it is obvious it was Windows which did not follow ACPI spec, nothing can stop BIOS engineers from mis-using _REV as a mechanism to detect Linux in BIOS ASL codes.

It did not take long when ACPI Spec Work Group (ASWG), the owner and the publisher of ACPI spec, realised that many systems are mis-using _REV objects in their firmware. Eventually a spec contributer submitted a change to _REV object in ACPI 6.0 and later, and the _REV object is now defined as below:



After ACPI 6.0 was published, the corresponding changes are also made to Linux kernel - the evaluation of _REV now returns 2 in kernel 4.1 and later (for example, Ubuntu 15.10 with kernel 4.2).

In order to provide backward compatibility, a new kernel parameter "acpi_rev_override" was introduced for those who still want to use _REV = 5 in their BIOS ASL code, but most of major Linux distributions do not include this kernel parameter by default, and it should be used with cautions.

If you are using one of the systems with _REV in the firmware, you can develop a kernel patch to add your system to the _REV quirk list acpi_rev_dmi_table in drivers/acpi/blacklist.c as below, like the patch I submitted.

static struct dmi_system_id acpi_rev_dmi_table[] __initdata = {
        {
         .callback = dmi_enable_rev_override,
         .ident = "DELL XPS 13 (2015)",
         .matches = {
                      DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
                      DMI_MATCH(DMI_PRODUCT_NAME, "XPS 13 9343"),
                },
        },
        ...
  • DMI_SYS_VENDOR can be identified by sudo dmidecode -s system-manufacturer 
  • DMI_PRODUCT_NAME can be identified by sudo dmidecode -s system-product-name.

Thursday, June 4, 2015

Uitlity: Reading and Writing PCI, IO and memory with Linux Firmware Debug Kit (lfdk) in Linux

Many BIOS engineers are familiar with ru.exe in DOS and R/W everything in Windows.  One frequently asked question is whether there is an equivalent utility in Linux.  The answer is "sort of".

Linux Firmware Debug Kit (lfdk) provides read/write functions to PCI[1], IO and memory spaces.

lfdk's deb package is available but it is outdated. I usually download [3] and compile source code, and run ldfk manually in a terminal (ctrl + alt + T if you are running Ubuntu +Unity) as below:

  1. sudo apt-get install -f git libncurses-dev
  2. git clone https://github.com/alexhungce/lfdk1.git
  3. cd lfdk
  4. make
  5. cd bin
  6. sudo insmod ./lfdd_drv.ko
  7. sudo ./lfdk

A familiar screen will pop up


The navigation  is to use arrow keys and hotkeys L, M and I to different spaces as instructed on the bottom of the lfdk.



[1] Built-in lspci command can be used to read PCI configuration space, too
[2] This is tested in Ubuntu Linux 12.04 to 15.04.
[3] One can also download the source code from sourceforge.net above.