Leica TCS: make sure XML file is on used files list #4252
Conversation
There was a problem hiding this comment.
Note Series002.xml will appear at the end of the list, which at first glance looks like it should cause test failures due to the changes in #4172. However, the use of initFile(...) on the first TIFF file found in line 263 means that the initialized file in the end is not Series002.xml.
From the reader logic, the XML file is treated an optional file with additional metadata so appending it to the end of the used files list rather makes sense.
With the current release of Bio-Formats using the sample dataset listed in the description, the XML file is not included in the the used files list after initialization when selecting either a TIFF file of the XML file. With this change, the XML is added to the used files:
sbesson@Sebastiens-MacBook-Pro-3 bioformats % ./tools/showinf -nopix ~/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002.xml
Checking file format [Leica TCS TIFF]
Initializing reader
TCSReader initializing /Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002.xml
Reading IFDs
Populating metadata
Checking comment style
Populating OME metadata
unknown creation date format: 0:00:00 00:00:00
Initialization took 0.064s
Reading core metadata
filename = /Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
Used files:
/Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
/Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch01.tif
/Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002.xml
Series count = 1
Series #0 :
Image count = 2
RGB = false (1)
Interleaved = false
Indexed = true (false color, 8-bit LUT: 3 x 256)
Width = 512
Height = 512
SizeZ = 1
SizeT = 1
SizeC = 2
Tile size = 512 x 512
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCTZ (certain)
Pixel type = uint8
Valid bits per pixel = 8
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #1 <=> Z 0, C 1, T 0
Reading global metadata
Number of image files: 2
Reading metadata
When selecting the TIFF file as an input, the XML file is not returned by getUsedFiles, should this be the case?
sbesson@Sebastiens-MacBook-Pro-3 bioformats % ./tools/showinf -nopix ~/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
Checking file format [Leica TCS TIFF]
Initializing reader
TCSReader initializing /Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
Reading IFDs
Populating metadata
Checking comment style
Populating OME metadata
unknown creation date format: 0:00:00 00:00:00
Initialization took 0.065s
Reading core metadata
filename = /Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
Used files:
/Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch00.tif
/Users/sbesson/bf-data-repo/automated-tests/curated/leica-tcs/brian/Series002_z00_ch01.tif
Series count = 1
Series #0 :
Image count = 2
RGB = false (1)
Interleaved = false
Indexed = true (false color, 8-bit LUT: 3 x 256)
Width = 512
Height = 512
SizeZ = 1
SizeT = 1
SizeC = 2
Tile size = 512 x 512
Thumbnail size = 128 x 128
Endianness = intel (little)
Dimension order = XYCTZ (certain)
Pixel type = uint8
Valid bits per pixel = 8
Metadata complete = false
Thumbnail series = false
-----
Plane #0 <=> Z 0, C 0, T 0
Plane #1 <=> Z 0, C 1, T 0
Reading global metadata
Number of image files: 2
Reading metadata
One inline question about a possible redundant call.
| @@ -247,6 +247,7 @@ protected void initFile(String id) throws FormatException, IOException { | |||
| Arrays.sort(list); | |||
|
|
|||
| boolean isXML = checkSuffix(id, XML_SUFFIX); | |||
| if (isXML) xmlFile = l.getAbsolutePath(); | |||
There was a problem hiding this comment.
Is this effectively redundant with the call after initFile(new Location(parent, file).getAbsolutePath()) below?
In particular, the pinhole size may be present in the XML, but just hasn't been parsed yet.
|
This ended up being quite a bit more involved than I originally anticipated. Last 3 commits here should mean that the XML file ends up on the used files list no matter what. However, the XML file being missing from the list meant that it wasn't being parsed at all - so several fixes were needed to address a Given the larger scope, I would be more comfortable with this going into 8.1.0 than 8.0.1, so have updated the milestone accordingly. |
There was a problem hiding this comment.
Retested the latest changes using the ImageInfo utility:
- with Bio-Formats 8.0.1, only the TIFF files from the
curated/leica-tcs/briandataset are selected in the used files list independently of the file passed to setId (including the XML). With this PR included, the two TIFF files and the XML file are always listed in the used file list independently of the file passed tosetId - the XML file is always appended to the list with this PR which matches the fact this file only contains additional metadata parsed by the reader
- the core metadata (number of images, dimensions,...) as well the binary data is unmodified by the changes in this PR
- with this PR included, when a dataset includes some XML file, the metadata is much richer, including
Instrumentmetadata, additionalPixels,ChannelandPlaneattributes as well as extended original metadata:
--- 8.0.1.xml 2024-11-18 11:40:53
+++ tiff_with_pr.xml 2024-11-18 11:22:19
@@ -1,24 +1,3544 @@
<?xml version="1.0" encoding="UTF-8"?>
<OME xmlns="http://www.openmicroscopy.org/Schemas/OME/2016-06" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.openmicroscopy.org/Schemas/OME/2016-06 http://www.openmicroscopy.org/Schemas/OME/2016-06/ome.xsd">
- <Image ID="Image:0" Name="Series002_z00_ch00.tif">
- <Pixels BigEndian="false" DimensionOrder="XYCTZ" ID="Pixels:0" Interleaved="false" SignificantBits="8" SizeC="2" SizeT="1" SizeX="512" SizeY="512" SizeZ="1" Type="uint8">
- <Channel ID="Channel:0:0" SamplesPerPixel="1">
- <LightPath/>
- </Channel>
- <Channel ID="Channel:0:1" SamplesPerPixel="1">
- <LightPath/>
- </Channel>
- <MetadataOnly/>
- <Plane TheC="0" TheT="0" TheZ="0"/>
- <Plane TheC="1" TheT="0" TheZ="0"/>
- </Pixels>
- </Image>
- <StructuredAnnotations>
- <XMLAnnotation ID="Annotation:0" Namespace="openmicroscopy.org/OriginalMetadata">
- <Value>
- <OriginalMetadata>
- <Key>Number of image files</Key>
- <Value>2</Value>
+ <Instrument ID="Instrument:0">
+ <Microscope Model="TCS SP5" Type="Other"/>
+ <Laser ID="LightSource:0:0" LaserMedium="Other" Type="Other" Wavelength="1.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:1" LaserMedium="Other" Type="Other" Wavelength="458.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:2" LaserMedium="Other" Type="Other" Wavelength="476.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:3" LaserMedium="Other" Type="Other" Wavelength="488.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:4" LaserMedium="Other" Type="Other" Wavelength="496.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:5" LaserMedium="Other" Type="Other" Wavelength="514.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:6" LaserMedium="Other" Type="Other" Wavelength="561.0" WavelengthUnit="nm"/>
+ <Laser ID="LightSource:0:7" LaserMedium="Other" Type="Other" Wavelength="633.0" WavelengthUnit="nm"/>
+ <Detector ID="Detector:0:0" Type="PMT"/>
+ <Detector ID="Detector:0:1" Type="PMT"/>
+ <Objective Correction="UV" ID="Objective:0:0" Immersion="Oil" LensNA="1.25" Model="HCX PL APO CS" NominalMagnification="40.0" SerialNumber="11506251"/>
+ <Filter ID="Filter:0:0">
+ <TransmittanceRange CutIn="500.0" CutInUnit="nm" CutOut="550.0" CutOutUnit="nm"/>
+ </Filter>
+ <Filter ID="Filter:0:1">
+ <TransmittanceRange CutIn="570.0" CutInUnit="nm" CutOut="620.0" CutOutUnit="nm"/>
+ </Filter>
+ <Filter ID="Filter:0:2" Model="SP Mirror Channel 1">
+ <TransmittanceRange CutIn="500.0" CutInUnit="nm" CutOut="550.0" CutOutUnit="nm"/>
+ </Filter>
+ <Filter ID="Filter:0:3" Model="SP Mirror Channel 2">
+ <TransmittanceRange CutIn="570.0" CutInUnit="nm" CutOut="620.0" CutOutUnit="nm"/>
+ </Filter>
+ <Filter ID="Filter:0:4" Model="SP Mirror Channel 3">
+ <TransmittanceRange CutIn="650.0" CutInUnit="nm" CutOut="750.0" CutOutUnit="nm"/>
+ </Filter>
+ </Instrument>
+ <Image ID="Image:0" Name="Series002_z00_ch00.tif">
+ <AcquisitionDate>2007-11-05T14:53:42</AcquisitionDate>
+ <InstrumentRef ID="Instrument:0"/>
+ <ObjectiveSettings ID="Objective:0:0" RefractiveIndex="1.518"/>
+ <Pixels BigEndian="false" DimensionOrder="XYCTZ" ID="Pixels:0" Interleaved="false" PhysicalSizeX="0.445197265625" PhysicalSizeXUnit="µm" PhysicalSizeY="0.445197265625" PhysicalSizeYUnit="µm" PhysicalSizeZ="0.999928474358057" PhysicalSizeZUnit="µm" SignificantBits="8" SizeC="2" SizeT="1" SizeX="512" SizeY="512" SizeZ="1" Type="uint8">
+ <Channel ExcitationWavelength="488.0" ExcitationWavelengthUnit="nm" ID="Channel:0:0" Name="Leica/FITC" PinholeSize="67.9629318522202" PinholeSizeUnit="µm" SamplesPerPixel="1">
+ <LightSourceSettings Attenuation="0.33998656" ID="LightSource:0:3"/>
+ <DetectorSettings Gain="722.209506370642" ID="Detector:0:0" Offset="-1.41333333333333"/>
+ <LightPath>
+ <EmissionFilterRef ID="Filter:0:0"/>
+ </LightPath>
+ </Channel>
+ <Channel ExcitationWavelength="561.0" ExcitationWavelengthUnit="nm" ID="Channel:0:1" Name="Leica/TRITC" PinholeSize="67.9629318522202" PinholeSizeUnit="µm" SamplesPerPixel="1">
+ <LightSourceSettings Attenuation="0.4699994" ID="LightSource:0:6"/>
+ <DetectorSettings Gain="856.603341725795" ID="Detector:0:1" Offset="-1.64666666666666"/>
+ <LightPath>
+ <EmissionFilterRef ID="Filter:0:1"/>
+ </LightPath>
+ </Channel>
+ <MetadataOnly/>
+ <Plane DeltaT="0.570705165513197" DeltaTUnit="s" TheC="0" TheT="0" TheZ="0"/>
+ <Plane DeltaT="1.76244129032258" DeltaTUnit="s" TheC="1" TheT="0" TheZ="0"/>
+ </Pixels>
+ </Image>
+ <StructuredAnnotations>
+ <XMLAnnotation ID="Annotation:0" Namespace="openmicroscopy.org/OriginalMetadata">
...
+ <XMLAnnotation ID="Annotation:435" Namespace="openmicroscopy.org/OriginalMetadata">
+ <Value>
+ <OriginalMetadata>
+ <Key>Series002_z00_ch00.tif Application</Key>
+ <Value>LAS AF</Value>
</OriginalMetadata>
</Value>
</XMLAnnotation>The configuration changes for the associated dataset include some of these metadata changes (channel names, wavelengths, physical sizes) and should ensure we can flag future regressions in the future.
The code changes to the LeicaHandler are consistent with how other handlers are operating i.e. the parsing method populates private attributes rather than populating the store directly for plane exposure times and timestamps. It is the responsibility of the reader to retrieve these using getter methods and populate the store within the reader.
While LeicaTCSReader is the only implementation in Bio-Formats consuming LeicaHandler, only API-level concern is that this could break a third-party consumer of the API relying on the existing handler logic. An option to keep this as a backwards-compatible addition would be to deprecate the old behavior and add a new constructor with a flag allowing to control the plane metadata population. LeicaTCSReader could then use the new constructor to disable the store setters.
I think this is now addressed by 10b48d2. |
Fixes #4229 (cc @mabruce).
As indicated in the original issue,
showinf curated/leica-tcs/brian/Series002.xmlwithout this change will not includeSeries002.xmlon the used file list. With this change, it should be included on the used file list.Note
Series002.xmlwill appear at the end of the list, which at first glance looks like it should cause test failures due to the changes in #4172. However, the use ofinitFile(...)on the first TIFF file found in line 263 means that the initialized file in the end is notSeries002.xml.Adding to 8.0.1 since this should be safe for a patch release, but this could easily be moved to a later milestone.