QCOW file header echoed in output

[DiskyBusiness]

Hi -- In testing with both libqcow-20181227 package and the latest master state, I am finding the QCOW file header echoed at quasi-regular points in the output. This behavior is seen when accessing either the pyqcow.file object or via the qcowmount utility.

See the attached image for an example, where the top file is created using the qemu-img converter and the bottom is the output of dd-ing the /qcow1 created by qcowmount. The first occurrence is at 0x180000, with a number of header recurrences at 0xFFF2 intervals after that. There's then a gap but the pattern repeats at 0x280000

[joachimmetz]

Could you share the test file or method you've created it?

[DiskyBusiness]

Unfortunately the QCOW file was not a test one, so I can't share it. I'll try to find something I can share, though, assuming this issue isn't specific to this file for some reason

[joachimmetz]

Thx that would be useful in reproducing it.

assuming this issue isn't specific to this file for some reason

Could be, I'm traveling and don't have my test files available.

[DiskyBusiness]

Thank you for spending the time to look at this. I have a sanitized qcow2 disk file that preserves enough of the original QCOW tables to mount and replicate the issue when I read from the /qcow1 stub.

https://send.firefox.com/download/d0a1c019951f2d6a/#MmU3aiqtUsSh38mBW6Qnww

[joachimmetz]

00180000  51 46 49 fb 00 00 00 03  00 00 00 00 00 00 00 00  |QFI.............|
00180010  00 00 00 00 00 00 00 10  00 00 00 06 40 00 00 00  |............@...|
00180020  00 00 00 00 00 00 00 32  00 00 00 00 00 03 00 00  |.......2........|
00180030  00 00 00 00 00 01 00 00  00 00 00 01 00 00 00 00  |................|
00180040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00180060  00 00 00 04 00 00 00 68  00 00 00 00 00 00 00 00  |.......h........|
00180070  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|
*
00190000  51 46 49 fb 00 00 00 03  00 00 00 00 00 00 00 00  |QFI.............|
00190010  00 00 00 00 00 00 00 10  00 00 00 06 40 00 00 00  |............@...|
00190020  00 00 00 00 00 00 00 32  00 00 00 00 00 03 00 00  |.......2........|
00190030  00 00 00 00 00 01 00 00  00 00 00 01 00 00 00 00  |................|
00190040  00 00 00 00 00 00 00 00  00 00 00 00 00 00 00 00  |................|

libqcow_io_handle_read_file_header: signature                           : 0x514649fb
libqcow_io_handle_read_file_header: format version                      : 3

libqcow_internal_file_read_buffer_from_file_io_handle: current offset                           : 0x00180000
libqcow_internal_file_read_buffer_from_file_io_handle: level 1 table index                      : 0
libqcow_internal_file_read_buffer_from_file_io_handle: level 2 table file offset                : 0x8000000000040000

libqcow_cluster_table_read: cluster table reference: 000                : 0x8000000000040000
libqcow_cluster_table_read: cluster table reference: 001                : 0x800000004b390000
libqcow_cluster_table_read: cluster table reference: 002                : 0x8000000000220000
libqcow_cluster_table_read: cluster table reference: 003                : 0x800000004b310000
libqcow_cluster_table_read: cluster table reference: 004                : 0x80000000002e0000
libqcow_cluster_table_read: cluster table reference: 005                : 0x8000000079ff0000
libqcow_cluster_table_read: cluster table reference: 006                : 0x80000000bdd00000
libqcow_cluster_table_read: cluster table reference: 007                : 0x00000000

libqcow_internal_file_read_buffer_from_file_io_handle: level 2 table index                      : 24
libqcow_internal_file_read_buffer_from_file_io_handle: cluster block file offset                : 0x00000001

libqcow_cluster_table_read: cluster table reference: 023                : 0x8000000000120000
libqcow_cluster_table_read: cluster table reference: 024                : 0x00000001

[joachimmetz]

The 0x00000001 points to the qcow file header so that explains the repeated header. The question now is what is cluster block file offset supposed to represent.

https://people.gnome.org/~markmc/qcow-image-format.html
https://git.qemu.org/?p=qemu.git;a=blob;f=block/qcow2.c;h=d507ee0686f0b295f6a655dfe14be87f03c138fd;hb=refs/heads/master

[joachimmetz]

https://git.qemu.org/?p=qemu.git;a=blob;f=block/qcow2.c;h=d507ee0686f0b295f6a655dfe14be87f03c138fd;hb=refs/heads/master#l2034

https://git.qemu.org/?p=qemu.git;a=blob;f=block/qcow2.h;hb=677746b39f068f4dc7658bcb13274124c2108125#l72

72 #define QCOW_OFLAG_ZERO (1ULL << 0)

https://git.qemu.org/?p=qemu.git;a=blob;f=block/qcow2.h;hb=677746b39f068f4dc7658bcb13274124c2108125#l530

 530     } else if (l2_entry & QCOW_OFLAG_ZERO) {
 531         if (l2_entry & L2E_OFFSET_MASK) {
 532             return QCOW2_CLUSTER_ZERO_ALLOC;
 533         }
 534         return QCOW2_CLUSTER_ZERO_PLAIN;
 535     } else if (!(l2_entry & L2E_OFFSET_MASK)) {

[joachimmetz]

I need to double check but the QCOW_OFLAG_ZERO could be something introduced in qcow format version 3. This flag is currently not handled by libqcow.