Utility functions for parsing user catalog files for skypac¶
Module for parsing @-files
or user input strings for use
by stsci.skypac
module.
- Authors
Mihai Cara
- License
- class stsci.skypac.parseat.FileExtMaskInfo(default_ext=('SCI', '*'), default_mask_ext=0, clobber=False, doNotOpenDQ=False, fnamesOnly=False, im_fmode='update', dq_fmode='readonly', msk_fmode='readonly')[source]¶
A class that holds image, dq, user masks, and extensions to be used with these files. It is designed to facilitate keeping track of user input in catalog files.
This class is intended to be used primarily for functions such as
parse_at_line()
and other related functions as a return value. It is also used to initializeskypac.skyline.SkyLine
objects.FileExtMaskInfo
was designed to be used in a specific ordered workflow. Below is a typical use of this class:Initialize the object with the desired settings for default extensions to be used with the files (when a specific extension for a file is not provided) and the open modes for the files;
Add image file using
image()
;Add image’s extension(s) using
append_ext()
;[Optional; can be performed at any later stage] Add DQ file and extensions using
DQimage()
anddqext()
methods;Append mask files and extensions using
append_mask()
;[Optional] Finalize the
FileExtMaskInfo()
object.
- Parameters
- default_ext: int, tuple, optional
Default extension to be used with image files that to not have an extension specified.
- default_mask_ext: int, tuple, optional
Default extension to be used with image mask files that to not have an extension specified.
- clobber: bool, optional
If a file being appended is in GEIS or WAIVER FITS format, should any existing MEF files be overwritten?
- doNotOpenDQ: bool, optional
Should the DQ files be oppened when simultaneously with the image files?
- fnamesOnly: bool, optional
Return file names only, or open the files and return
ImageRef
objects?- im_fmode: str, optional
File mode to be used to open image FITS file. See
astropy.io.fits.open
for more details.- dq_fmode: str, optional
File mode to be used to open DQ FITS file. This is valid only if the DQ model of the image file is ‘external’ (see documentation for
ImageRef
for more details). For ‘intrinsic’ DQ model the DQ files will use the same setting as forim_fmode
.- msk_fmode: str, optional
File mode to be used to open mask files.
- Attributes
- clobber: bool
If a file being appended is in GEIS or WAIVER FITS format, should any existing MEF files be overwritten?
- dq_bits: int
Bitmask specifying what pixels in the mask should be removed (or kept) with the precise interpretation being left to the user. This flag is not used by this class but was designed to be passed to other functions that will use
FileExtMaskInfo
.
- property DQimage¶
DQ image (file or
ImageRef
object depending on thefnamesOnly
value).
- append_ext(ext)[source]¶
Append extensions to the list of “selected” extensions for the image file.
Note
This function appends the extensions. If it is desired to set the extensions, use
replace_ext()
instead.- Parameters
- ext: int, tuple, None, list
Extension specification: None, an integer extension number, a tuple (extension number, extension version) where extension version can be
'*'
which will be replaced with the extension versions of all extensions having given extension name. If ext is None, it will be replaced with the default extension specification set during the initialization of theFileExtMaskInfo
object.
- append_mask(mask, ext, mask_stat=None)[source]¶
Append a mask image and its extension(s).
Note
Mask files and extensions are kept in ordered lists and their order is significant: the first mask file-extension pair is associated with the first extension of the science image file set with
append_ext()
and so on.- Parameters
- mask: str, ImageRef, None
Mask image file. Can be a string file name, an
ImageRef
object (only iffnamesOnly
isFalse
), orNone
(to act as a place holder in the ordered list of extensions).- ext: int, tuple, None, list
Extension specification: None, an integer extension number, a tuple (extension number, extension version) where extension version can be
'*'
which will be replaced with the extension versions of all extensions having given extension name. If ext is None, it will be replaced with the default extension specification for mask images set during the initialization of theFileExtMaskInfo
object.- mask_stat: `os.stat_result`, optional
An
os.stat_result
structure for the inputmask
file. IfNone
, thenappend_mask()
will computestat
for the inputmask
file.
- Raises
- RuntimeError
Raised if attempting to add masks when the science image was not yet set.
- AssertionError
- TypeError
Raised if
mask
is anImageRef
object butfnamesOnly
isTrue
or ifmask
argument is of incorrect type.- ValueError
If
mask
is anImageRef
, it must not be closed.
- convert2ImageRef()[source]¶
Replace any existing file names with opened
ImageRef
objects and change thefnamesOnly
property toFalse
.Note
The
finalized
property will not be modified.Warning
The
FileExtMaskInfo
must not have been finalized (finalized
isFalse
) and must contain file names only (fnamesOnly
isTrue
).- Raises
- AssertionError
Raised if
finalized
isTrue
orfnamesOnly
isFalse
.
See also
- property count¶
Number of extensions associated with the image file.
- property dqext¶
FITS extensions associated with the DQ file.
- property fext¶
FITS extensions associated with the image file.
- finalize(toImageRef=False)[source]¶
Finalize the object by trimming or extending mask image lists to match the number of science image extensions.
In principle, the number of mask files and their extensions need not be equal to the number of extensions specified for the science image. If the number of masks/extensions is smaller than the number of science extensions, the list of mask extensions will be appended with None (if
fnamesOnly
isTrue
) or dummyImageRef
(iffnamesOnly
isFalse
) until the number of mask extensions is equal to the number of science image extensions. If the number of mask extensions is larger than the number of science image extensions, the list of mask extensions will be trimmed to match the number of science image extensions. The trimmed out mask files (if represented byImageRef
) will be “released”.
- property finalized¶
Is the
FileExtMaskInfo
object finalized?
- property fnamesOnly¶
Was the
FileExtMaskInfo
initialized to return file names or theImageRef
objects?
- property image¶
Image file name or the associated
ImageRef
object (depending on thefnamesOnly
value).Note
Setting the image will re-initialize
FileExtMaskInfo
. All previous settings will be lost and previously attached files will be released/deleted.
- property mask_images¶
Mask image file names or
ImageRef
object depending on thefnamesOnly
value.
- property maskext¶
FITS extensions associated with the mask files.
- release_all_images()[source]¶
Release all images if
fnamesOnly
isFalse
and replace any existingImageRef
with their original file names.Note
This will set the
fnamesOnly
property toTrue
and thefinalized
property toFalse.
See also
- stsci.skypac.parseat.parse_at_file(fname, default_ext=('SCI', '*'), default_mask_ext=0, clobber=False, fnamesOnly=False, doNotOpenDQ=False, match2Images=None, im_fmode='update', dq_fmode='readonly', msk_fmode='readonly', logfile=None, verbose=False)[source]¶
This function is similar to
parse_at_line()
, the main difference being that is can parse multiple (EOL terminated) lines of the format described in the documentation forparse_at_line()
.Below we describe only differences bewtween this function and
parse_at_line()
.- Parameters
- fname: str
File name of the catalog file.
- match2Images: list of str, list of ImageRef, None, optional
List of file names or ImageRef objects whose mask specifications are to be read from the catalog file. Mask specifications for other files in the catalog that do not match the files in the
match2Images
list will be ignored. Ifmatch2Images
isNone
, then all files from the catalog will be read.- logfile: str, file, MultiFileLog, None, optional
Specifies the log file to which the messages should be printed. It can be a file name, a file object, a
MultiFileLog
object, orNone
.
- Returns
- list
Returns a list of filenames if
fnamesOnly
isTrue
or a list ofFileExtMaskInfo
objects iffnamesOnly
isFalse
.
- stsci.skypac.parseat.parse_at_line(fstring, default_ext=('SCI', '*'), default_mask_ext=0, \ clobber=False, fnamesOnly=False, doNotOpenDQ=False, \ im_fmode='update', dq_fmode='readonly', msk_fmode='readonly', verbose=False)[source]¶
Parse a line from a catalog file containing a science image file and extension specifications and optionally followed by a comma-separated list of mask files and extension specifications (or
None
).File names will be stripped of leading and trailing white spaces. If it is essential to keep these spaces, file names may be enclosed in single or double quotation marks. Quotation marks may also be required when file names contain special characters used to separate file names and extension specifications:
,[]{}
Extension specifications must follow the file name and must be delimited by either square or curly brackets. Curly brackets allow specifying multiple comma-separated extensions: integer extension numbers and/or tuples (‘ext name’, ext version).
- Some possible ways of specifying extensions:
[1] – extension number
[‘sci’,2] – extension name and version
{1,4,(‘sci’,3)} – multiple extension specifications, including tuples
{(‘sci’,*)} – wildcard extension versions (i.e., all extensions with extension name ‘sci’)
[‘sci’] – equivalent to [‘sci’, 1]
{‘sci’} – equivalent to {(‘sci’,*)}
For extensions in the science image for which no mask file is provided, the corresponding mask file names may be omitted (but a comma must still be used to show that no mask is provided in that position) or
None
can be used in place of the file name. NOTE:'None'
(in quotation marks) will be interpreted as a file named'None'
.- Some examples of possible user input:
image1.fits{1,2,('sci',3)},mask1.fits,,mask3.fits[0]
In this case:
image1.fits[1]
is associated withmask1.fits[0]
;image1.fits[2]
does not have an associated mask;image1.fits['sci',3]
is associated withmask3.fits[0]
.– Assume
image2.fits
has 4 ‘SCI’ extensions:image2.fits{'sci'},None,,mask3.fits
In this case:
image2.fits['sci', 1]
andimage2.fits['sci', 2]
andimage2.fits['sci', 4]
do not have an associated mask;image2.fits['sci', 3]
is associated withmask3.fits[0]
Note
Wildcard extension version in extension specification can be expanded only when
fnamesOnly
isFalse
.- Parameters
- fstring: str
A comma-separated string describing the image file name and (optionally) followed by the extension specifier (e.g., [sci,1,2], or [sci]). The image file name may be followed (comma-separated) by optional mask file names (and their extension specifier).
File and extension names may NOT contain leading and/or trailing spaces, commas, and/or square or curly brakets.
- default_ext: int, tuple, optional
Default extension to be used with image files that to not have an extension specified.
- default_mask_ext: int, tuple, optional
Default extension to be used with image mask files that to not have an extension specified.
- fnamesOnly: bool, optional
Return file names only, or open the files and return
ImageRef
objects?- doNotOpenDQ: bool, optional
Should the DQ files be oppened when simultaneously with the image files?
- im_fmode: str, optional
File mode to be used to open image FITS file. See
astropy.io.fits.open
for more details.- dq_fmode: str, optional
File mode to be used to open DQ FITS file. This is valid only if the DQ model of the image file is ‘external’ (see documentation for
ImageRef
for more details). For ‘intrinsic’ DQ model the DQ files will use the same setting as forim_fmode
.- msk_fmode: str, optional
File mode to be used to open mask files.
- verbose: bool, optional
Specifies whether to print warning messages.
- Returns
- FileExtMaskInfo
A
FileExtMaskInfo
object.
- Raises
- ValueError
Input argument ‘fstring’ must be a Python string.
Input argument ‘fstring’ contains either unbalanced or nested square brackets.
Extension specification must be preceeded by a valid image file name.
- stsci.skypac.parseat.parse_cs_line(csline, default_ext=('SCI', '*'), clobber=False, fnamesOnly=False, doNotOpenDQ=False, im_fmode='update', dq_fmode='readonly', msk_fmode='readonly', logfile=None, verbose=False)[source]¶
This function is similar to
parse_at_line()
, the main difference being the content of the input string: a list of comma-separated science image file names. No masks can be specified and file names must be valid (i.e.,None
is not allowed). Extension specifications are allowed and must folow the same sintax as described forparse_at_line()
.Below we describe only differences bewtween this function and
parse_at_line()
.- Parameters
- csline: str
User input string that needs to be parsed containing one of the following:
a comma-separated list of valid science image file names (see note below) and (optionally) extension specifications, e.g.:
'j1234567q_flt.fits[1], j1234568q_flt.fits[sci,2]'
;an @-file name, e.g.,
'@files_to_match.txt'
.
Note
Valid science image file names are:
file names of existing FITS, GEIS, or WAIVER FITS files;
partial file names containing wildcard characters, e.g.,
'*_flt.fits'
;Association (ASN) tables (must have
_asn
, or_asc
suffix), e.g.,'j12345670_asn.fits'
.
Warning
@-file names MAY NOT be followed by an extension specification.
Warning
If an association table or a partial file name with wildcard characters is followed by an extension specification, it will be considered that this extension specification applies to each file name in the association table or each file name obtained after wildcard expansion of the partial file name.
- logfile: str, file, MultiFileLog, None, optional
Specifies the log file to which the messages should be printed. It can be a file name, a file object, a MultiFileLog object, or None.
- Returns
- list
Returns a list of filenames if
fnamesOnly
isTrue
or a list ofFileExtMaskInfo
objects iffnamesOnly
isFalse
.