Merge pull request #60 from gaia-dpci/remove-in-progress-features

Prepare version with speed improvements

.github/workflows/python-package.yml

@@ -11,11 +11,12 @@ jobs:
   build:
 
     runs-on: ${{ matrix.os }}
+    timeout-minutes: 70  # Set the timeout value in minutes
     strategy:
       fail-fast: false
       matrix:
-        python-version: ["3.6", "3.7", "3.8", "3.9", "3.10"]
-        os: [ubuntu-20.04, macOS-latest, windows-latest]
+        python-version: ["3.7", "3.8", "3.9", "3.10", "3.11"]
+        os: [ubuntu-latest, macOS-latest, windows-latest]
 
     steps:
     - uses: actions/checkout@v3
@@ -27,7 +28,6 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         python -m pip install flake8 pytest
-        pip install -r requirements.txt
     - name: Install package
       run: |
         pip install -e .
@@ -39,4 +39,4 @@ jobs:
         flake8 . --count --exit-zero --max-complexity=10 --max-line-length=127 --statistics
     - name: Test with pytest
       run: |
-        pytest -s
+        ./run_tests.sh --no-plotter

.gitignore

@@ -3,6 +3,7 @@
 .coverage
 .eggs/
 .env/
+.env10/
 .idea/
 .pytest_cache/
 .svn/

docs/source/_static/css/custom.css

@@ -5,3 +5,15 @@ th p {
 .section #basic-2-flip-flop-synchronizer{
      text-align:justify;
 }
+
+table td:nth-child(1) {
+    background-color: #FFD700;  /* Yellow */
+}
+
+table td:nth-child(2) {
+    background-color: #87CEEB;  /* Sky Blue */
+}
+
+table td:nth-child(3) {
+    background-color: #90EE90;  /* Light Green */
+}

docs/source/caveats.rst

@@ -7,37 +7,50 @@ Caveats / Known issues
 Plotter
 -------
 
-The output DataFrames contain metadata indicating which type of spectra they contain. This metadata
-can be accessed through the attribute :python:`attrs`. The plotter tool relies on this information
-to know which type of plot to generate.
+The output DataFrames contain metadata indicating which type of spectra they contain. This metadata can be accessed
+through the attribute :python:`attrs`. The plotter tool relies on this information to know which type of plot to generate.
 
-Pandas does not yet have a robust method of propagating metadata attached to DataFrames,
-so if some operations are performed over the DataFrame, this information may be lost.
-Luckily, most common operations preserve metadata.
-
-The plotter can only receive pandas DataFrames and not pandas Series, which may be important when
-selecting a particular row of a DataFrame.
-
-For example, if :python:`df` is a DataFrame returned by GaiaXPy corresponding to either the calibrator or converter output,
-and we want to select row :python:`0` and the use :python:`plot_spectra` on it:
-
-.. code-block:: python
-
-    df.iloc[0] # Will return a Pandas Series, not accepted by plot_spectra
-    df.iloc[[0]] # Will return a Pandas DataFrame, accepted by plot_spectra
+Pandas does not yet have a robust method of propagating metadata attached to DataFrames. As a result, if certain
+operations are applied to the DataFrame, this information may be lost which will cause the plotter to fail. Luckily,
+most common operations preserve metadata.
 
 Minimum requirements
 --------------------
 
-GaiaXPy works with Python 3.6 and later on Linux, MacOS, and Windows. If a problem is found, it is recommended to report it by creating a `GitHub issue <https://github.com/gaia-dpci/GaiaXPy/issues>`_.
+GaiaXPy works with Python 3.7 and later on Linux, MacOS, and Windows. If a problem is found, it is recommended to report
+it by creating a `GitHub issue <https://github.com/gaia-dpci/GaiaXPy/issues>`_.
+
+Previous versions of GaiaXPy were compatible with Python 3.6.
 
 Filters
 -------
 
-Since GaiaXPy was originally published, two issues with photometric filters have been found. The affected filters are **SDSS** and **PanSTARRS1_Std**.
+Since GaiaXPy was originally published, two issues with photometric filters have been found. The affected filters are
+**SDSS** and **PanSTARRS1_Std**.
 
-**SDSS**: The SDSS filters (available as PhotometricSystem.SDSS) that were originally published have been replaced with those defined in `Doi et al. 2010, AJ, 141, 47 <https://ui.adsabs.harvard.edu/abs/2010AJ....139.1628D/abstract>`_. Note that the standardised version (PhotometricSystem.SDSS_Std) was already based on Doi et al. 2010, AJ, 141, 47 and is therefore unchanged with respect to previous versions of GaiaXPy.
+**SDSS**: The SDSS filters (available as PhotometricSystem.SDSS) that were originally published have been replaced with
+those defined in `Doi et al. 2010, AJ, 141, 47 <https://ui.adsabs.harvard.edu/abs/2010AJ....139.1628D/abstract>`_.
+Note that the standardised version (PhotometricSystem.SDSS_Std) was already based on Doi et al. 2010, AJ, 141, 47 and is
+therefore unchanged with respect to previous versions of GaiaXPy.
 
-**PanSTARRS1_Std**: Due to a bug in GaiaXPy, the synthetic photometry for the standardized PanSTARRS1 y band (contained in the fields y_ps1_flux, y_ps1_flux_error and y_ps1_mag) has been generated without applying the flux offset mitigating the systematic effect at the faint end due to background issues (also referred to as hockey-stick, see Section 2.2.1 and equation 13 in `Gaia Collaboration, Montegriffo, et al., 2022 <https://ui.adsabs.harvard.edu/abs/2022arXiv220606215G/abstract>`_). The offset should have been applied to the synthetic flux and then propagated to the flux error and magnitude. However, no offset was being applied.
+**PanSTARRS1_Std**: Due to a bug in GaiaXPy, the synthetic photometry for the standardized PanSTARRS1 y band (contained
+in the fields y_ps1_flux, y_ps1_flux_error and y_ps1_mag) has been generated without applying the flux offset mitigating
+the systematic effect at the faint end due to background issues (also referred to as hockey-stick, see Section 2.2.1 and
+equation 13 in `Gaia Collaboration, Montegriffo, et al., 2022 <https://ui.adsabs.harvard.edu/abs/2022arXiv220606215G/abstract>`_).
+The offset should have been applied to the synthetic flux and then propagated to the flux error and magnitude. However,
+no offset was being applied.
 
 Both issues were **fixed in GaiaXPy version 1.2.4**.
+
+OpenSSL / LibreSSL
+------------------
+In some cases, an incompatibility problem with OpenSSL/LibreSSL may arise. This is due to a problem with recent versions
+of the urllib3 (as of May 2023) used by the requests library which is required to connect to the Gaia Archive.
+The error message contains the sentence: "urllib3 v2.0 only supports OpenSSL 1.1.1+"
+The problem can be resolved by installing urllib3 version 1.26.6:
+
+.. code-block:: python
+
+    pip install urllib3==1.26.6
+
+More information `here <https://github.com/urllib3/urllib3/issues/3020>`_.

docs/source/conf.py

@@ -4,14 +4,15 @@
 
 # -- Path setup --------------------------------------------------------------
 
+import os
 # If extensions (or modules to document with autodoc) are in another directory,
 # add these directories to sys.path here. If the directory is relative to the
 # documentation root, use os.path.abspath to make it absolute, like shown here.
 import re
-import os
 import sys
+
 import numpy as np
-import sphinx_rtd_theme
+
 sys.path.insert(0, os.path.abspath('..'))
 sys.path.append('gaiaxpy')
 from gaiaxpy import __version__
@@ -53,15 +54,14 @@ def setup(app):
 
 # -- Project information -----------------------------------------------------
 project = u'GaiaXPy'
-copyright = u'2022, DPAC, CU5, DPCI'
+copyright = u'2023, DPAC, CU5, DPCI'
 author = u'Francesca De Angeli, Paolo Montegriffo, Lovro Palaversa, Daniela Ruz-Mieres'
 
 # The short X.Y version
 version = f'{__version__} (latest)'
 # The full version, including alpha/beta/rc tags
 release = __version__
 
-
 # -- General configuration ---------------------------------------------------
 
 # If your documentation needs a minimal Sphinx version, state it here.
@@ -110,7 +110,6 @@ def setup(app):
 # The name of the Pygments (syntax highlighting) style to use.
 pygments_style = 'sphinx'
 
-
 # -- Options for HTML output -------------------------------------------------
 
 # The theme to use for HTML and HTML Help pages.  See the documentation for
@@ -136,14 +135,12 @@ def setup(app):
     'titles_only': False,
 }
 
-#html_theme_path = [sphinx_rtd_theme.get_html_theme_path()]
-
 # Add any paths that contain custom static files (such as style sheets) here,
 # relative to this directory. They are copied after the builtin static files,
 # so a file named "default.css" will overwrite the builtin "default.css".
 html_static_path = ['_static']
 
-html_logo = "_static/images/gaiaLogo.png"
+html_logo = "_static/images/GaiaXPy.png"
 html_favicon = html_logo
 
 html_css_files = [
@@ -166,7 +163,6 @@ def setup(app):
 # Output file base name for HTML help builder.
 htmlhelp_basename = 'GaiaXPydoc'
 
-
 # -- Options for LaTeX output ------------------------------------------------
 
 latex_elements = {
@@ -203,7 +199,6 @@ def setup(app):
      u'Francesca De Angeli, Paolo Montegriffo, Lovro Palaversa, Daniela Ruz-Mieres', 'manual'),
 ]
 
-
 # -- Options for manual page output ------------------------------------------
 
 # One entry per manual page. List of tuples
@@ -213,19 +208,18 @@ def setup(app):
      [author], 1)
 ]
 
-
 # -- Options for Texinfo output ----------------------------------------------
 
 # Grouping the document tree into Texinfo files. List of tuples
 # (source start file, target name, title, author,
 #  dir menu entry, description, category)
 texinfo_documents = [
     (master_doc, 'GaiaXPy', u'GaiaXPy Documentation',
-     author, 'GaiaXPy', 'Tools to generate sampled Gaia BP/RP mean spectra from the data served via the Gaia DR3 archive.',
+     author, 'GaiaXPy',
+     'Tools to generate sampled Gaia BP/RP mean spectra from the data served via the Gaia DR3 archive.',
      'Miscellaneous'),
 ]
 
-
 # -- Options for Epub output -------------------------------------------------
 
 # Bibliographic Dublin Core info.
@@ -243,5 +237,4 @@ def setup(app):
 # A list of files that should not be packed into the epub file.
 epub_exclude_files = ['search.html']
 
-
 # -- Extension configuration -------------------------------------------------

docs/source/description.rst

@@ -4,7 +4,7 @@ About the package
 General description
 -------------------
 
-GaiaXPy is a Python library to facilitate handling Gaia BP/RP spectra as distributed from the `Gaia archive <https://gea.esac.esa.int/archive/>`_.
+GaiaXPy is a Python library to facilitate handling Gaia BP/RP spectra as distributed from the `Gaia Archive <https://gea.esac.esa.int/archive/>`_.
 
 BP/RP (often shortened as XP) spectra became available for the first time in Gaia Data Release 3 (DR3).
 In their first release, only source **mean spectra** are available: these are spectra that have been generated from a number of single observations of the same object. **Epoch spectra**, i.e. spectra consisting of one single observation, will become available in future releases.
@@ -32,6 +32,7 @@ The GaiaXPy library will grow in time. For Gaia DR3 the following functionalitie
 - **Conversion** of mean spectra from a continuous representation to a sampled spectrum, optionally on a user-defined grid of positions. The output spectrum will be in the internal system (if an absolute spectrum is required, please use the Calibration functionality).
 - **Generation** of synthetic photometry in bands that are covered by the BP/RP wavelength range. Several sets of filters are included in the package.
 
-A further functionality is being developed and will be released soon:
+Two further functionalities are being developed and will be released soon:
 
+- **Detection** of lines and extrema in mean spectra.
 - **Simulation** of Gaia BP/RP internally calibrated spectra starting from an input Spectral Energy Distribution (SED) given the Gaia DR3 instrument model.