Data conversion from Thermo Fisher EBSD detector/software

[reinis-ignatans]

Dear All,

Does anyone have some experience/knowledge of converting EBSD data produced by Thermo Fisher's Pathfinder software to hdf5 or other "more open" data type so it can be loaded in KikuchiPy. There is no meaningful export possibility natively in Pathfinder (well, unless you want to export a single EBSD pattern thousands of times by clicking...). What I have deduced is that the patterns are stored in *.pat file, however googling does not yield anything on the format or the structure of the file.

Best wishes and have great diffraction patterns!
Reinis

[hakonanes]

Hi Reinis,

Thank you for bringing this up. I knew Thermo Fisher had an EBSD system, but hadn't seen references to it or patterns from it. That's why we don't have a reader for them.

If you can share a small .pat file and the settings (map shape, pattern shape etc.) and any accompanying files (with settings and orientation data), we are closer to getting a TF reader.

There is a longer term plan to move existing readers and writers from kikuchipy to RosettaSciIO. This hasn't been done yet due to limited resources. I suggest to implement a reader in kikuchipy for now.

[reinis-ignatans]

Hi,

Thanks for such a swift reply. I will gather smaller dataset on nickel standard tomorrow for this purpose. At the moment I only have quite huge datasets on my hands.

Best,
R

[hakonanes]

No rush, as I'm very busy this and next week.

[reinis-ignatans]

Hello,
Please find the data set here:
Thermo Fisher EBSD

Measured material is Nickel, the map contains a few grain boundaries.

The *.zip file contains all the files the software creates during measurement. I have added an additional folder containing screenshots from the software with various parameters (who knows, what is needed, right). I have added four EBSD pattern exports from the map's four corners.

The data have been acquired with 25 kV, 6.4 nA, sample tilt 70 deg, working distance 6.1 mm (more precise value is closer to 6.06 mm). EBSD map is 64 x 42 pixels. Electron image is 512 x 340 pixels. EBSD patterns (I think) are acquired with 640 x 400 pixels with 2 x 2 binning. There is also pattern center coordinates in some of the screenshots.

Thanks and best wishes,
R

[hakonanes]

Thank you for providing a small test dataset and detailed extra information, Reinis!

I only had time for a quick look now. I found the following:

1. Nav shape: (n rows, n cols) = (42, 64); sig shape: (s rows, s cols) = (200, 320)
2. File "EBSD Display Properties.ini" line 91 "Num Bins=4096" (under "EBSD Image Histogram" part) tells me pattern intensities are stored as 16-bit integers (I assume unsigned)
3. Pattern file ("KikuchiPy(2).pat") size: 344 213 529 bytes

"Known" bytes are then 42 * 64 * 200 * 320 * 2 = 344 064 000.

This means we have 344 213 529 - 344 064 000 = 149 529 "unknown" bytes. I don't have time to look at the pattern file now. But we should with the data you've provided be able to find the starting byte for the first pattern and then the systematic stride to read the other patterns.

If the Thermo Fisher file is similar to Oxford's binary file, there's a file header (file version, map shape, pattern shape, other parameters like the bit depth) and then a pattern header, the pattern, a pattern footer, etc. until the end. If you (or someone else) want to have a go at it, the Oxford binary reader can be a good place to start.

[hakonanes]

Figured it out. The structure is as follows (in byte order):

File header:

No. bytes	Type	What	Value
4	int32, i	File version	1
4	int32, i	Map columns	64
4	int32, i	Map rows	42
4	int32, i	Pattern columns	320
4	int32, i	Pattern rows	200
2	uint16, H	Bit depth per pattern	65 535
1	bool, ?	Unknown	True
2	uint16, H	Bit depth per pattern	65 535
128 000	uint16, H	Background pattern	Intensities

Pattern header:

No. bytes	Type	What	Value
4	int32, i	Beam x	0 (for first pattern)
4	int32, i	Beam y	0 (for first pattern)
128 000	uint16, H	Pattern	Raw intensities

All bytes:

• Header: 25
• Background pattern: 128 000
• Pattern header: 8 * 2 688
• Pattern: 128 000 * 2 688
• Sum: 344 213 529

[reinis-ignatans]

Oh, fantastic!
I just could not stand the feeling of not having easy access to the acquired data.

Thanks!