Space Telescope Environment (`stenv`)¶

stenv is an installable Conda environment maintained by the Space Telescope Science Institute (STScI) in Baltimore, Maryland. This environment provides tools and utilities required to process and analyze data from the Hubble Space Telescope (HST), James Webb Space Telescope (JWST), and others.

Frequently Asked Questions¶

`stenv` doesn’t build on my system; what do I do?¶

You can use the environment definition YAML file (environment.yaml) in the root of the repository:

conda

conda env create -n stenv -f https://raw.githubusercontent.com/spacetelescope/stenv/main/environment.yaml

mamba

mamba env create -n stenv -f https://raw.githubusercontent.com/spacetelescope/stenv/main/environment.yaml

This environment is unpinned, meaning it may take some time to resolve dependency versions. Additionally, the resulting package versions may not have been tested for your platform.

Warning

stenv does not currently support a native Windows installation. To build stenv on Windows, see Windows Support.

Why isn’t _____ package in `stenv`?¶

Not all STScI packages are included in the base stenv environment; some packages are not supported and / or deprecated, and some are deemed too niche (or dependent on too many extra packages) to be included for all users.

To install a package in your local environment, you can use pip install while the environment is activated:

conda

conda activate stenv
pip install <package_name>

mamba

mamba activate stenv
pip install <package_name>

To request that a new package be added to stenv (environment.yaml) for all users, see Adding a package to stenv.

What about Astroconda?¶

Astroconda, historically maintained by STScI as a Conda software channel, provides data analysis tools and pipelines via the Conda package management system.

Warning

Astroconda is no longer supported as of February 1st, 2023.

stenv supersedes Astroconda as a STScI software distribution; it supports most of the packages in Astroconda, works with all current versions of Python, and provides a common environment for both the Hubble Space Telescope (HST) and James Webb Space Telescope (JWST) pipelines. Additionally, while Astroconda primarily uses Conda recipes to build and serve packages, which need to be updated separately from PyPI releases, stenv draws most of its packages directly from PyPI with pip (though it still requires use of a Conda environment for hstcal and fitsverify, which are provided by conda-forge).

Developer Notes¶

stenv consists of several parts:

an unpinned Conda environment definition YAML file environment.yaml
a GitHub Actions CI workflow that automatically builds and tests the environment on several platforms
regular GitHub releases with attached constrained Conda environment definition YAML files for every tested platform

`environment.yaml`¶

# this is the base unpinned environment for ``stenv``; to manually resolve an environment from this file, run the following: 
# conda env create -n stenv -f https://raw.githubusercontent.com/spacetelescope/stenv/main/environment.yaml 

channels:
  - conda-forge

dependencies:
  - astropy
  - dask
  - fitsverify
  - hstcal
  - h5py
  - ipython
  - jupyter
  - numpy
  - pip
  - pytables
  - python>=3.10
  - pytest
  - pytest-xdist
  - scipy
  - scikit-image
  - sphinx
  - pip:
      - acstools
      - asdf
      - calcos
      - ccdproc
      - ci_watson
      - costools
      - crds
      - drizzlepac
      - ginga
      - jwst
      - nictools
      - pysynphot
      - reftools
      - stcal
      - stdatamodels
      - stistools
      - stsynphot
      - stregion
      - synphot
      - wfc3tools
      - wfpc2tools

To build an environment from this unpinned environment definition, you may run the following:

conda

conda env create -n stenv -f https://raw.githubusercontent.com/spacetelescope/stenv/main/environment.yaml

mamba

mamba env create -n stenv -f https://raw.githubusercontent.com/spacetelescope/stenv/main/environment.yaml

Adding a package to `stenv`¶

To request that a new package be added to stenv, please create a new issue in the repository.

Pipeline Releases¶

Note

A working conda installation (Miniconda, Mamba, Anaconda, etc.) is required (see Install Conda).

Warning

stenv does not support Python 2.

Warning

stenv does not support 32-bit operating systems.

Pipeline releases differ from the standard software stack and serve a different purpose. The release files, described below, are immutable snapshots of STScI operational software and can be used to replicate the environment used by STScI to perform mission-specific data processing. Be aware that upgrading packages with conda update is not recommended, as it will likely introduce unwanted bugs and / or break the environment all together.

If you have any questions, comments, or concerns related to pipeline releases, please feel free to contact help@stsci.edu

Installation¶

Pipeline release installations use the following conda create command format:

conda create -n demo_2016.1 --file http://ssb.stsci.edu/releases/hstdp/2016.1/hstdp-2016.1-linux-py35.0.txt
conda activate demo_2016.1

Warning

The URL shown in this example does not reflect the latest iteration available. Please consult the File URLs section to ensure you are installing the correct release.

File URLs¶

Select the URL that matches your intended platform and environment.

HST Data Processing (HSTDP)¶

HSTDP was previously known as OPUS.

Instructions for installation of each delivery may be found in the respective subdirectories of the releases repository: https://github.com/astroconda/astroconda-releases/tree/master/caldp

Warning

The repository is located within the Astroconda GitHub organization for legacy reasons. Astroconda is no longer supported as of February 1st, 2023.

Historical deliveries used an older naming convention: https://github.com/astroconda/astroconda-releases/tree/master/hstdp

Continuous Integration¶

This example BASH function provides a starting point for users intending to execute pipeline software from within a continuous integration environment. This installation method is unsupported and your mileage may vary. Use at your own risk.

function get_pipeline()
{
    # Do we have enough arguments?
    if [[ $# < 3 ]]; then
        echo "Not enough arguments."
        return 1
    fi

    # Setup basic argument list     & Example Input(s)
    local conda_env="$1"            # hst_env
    local name="$2"                 # hstdp, ...
    local build="$3"                # 2017.2, 2016.2 ...
    local python_version="$4"       # py[35, 27, ...]
    local iteration="$5"            # final | post[0, 1, 2, ...]

    # Detect platform
    local _platform=$(uname -s)
    local platform=""

    # Convert platform string to match file naming convention
    if [[ ${_platform} == Linux ]]; then
        platform="linux"
    elif [[ ${_platform} == Darwin ]]; then
        platform="osx"
    else
        echo "Unsupported platform: ${_platform}"
        return 1
    fi
    unset _platform

    # Handle optional arguments.
    if [[ -z ${python_version} ]]; then
        # Notice the "py" prefix and condensed version here
        python_version="py35"
    fi

    if [[ -z ${iteration} ]]; then
        iteration="final"
    fi

    # Assemble pipeline spec file URL
    local ac_root="http://ssb.stsci.edu/releases"
    local ac_base="${ac_root}/${name}/${build}"
    local ac_spec="${name}-${build}-${platform}-${python_version}.${iteration}.txt"
    local ac_url="${ac_base}/${ac_spec}"

    # Perform installation
    conda create -q -n "${conda_env}" --file "${ac_url}"
    return $?
}

#
# Usage example:
#

# Silently generate a pipeline environment called "hst_env"
get_pipeline hst_env hstdp 2017.2

# Enter environment
source activate hst_env

# ... do work ...
# EOF

Windows Support¶

stenv does not currently support Windows, as hstcal, fitsverify, and the jwst calibration pipeline are not built or tested on Windows platforms.

If you would like to run stenv on Windows, you can use the Windows Subsystem for Linux (WSL), an optional Windows feature that provides a functioning Linux terminal with access to the host operating system.

After installing your Linux system, install stenv by following the Getting Started instructions.

Windows Subsystem for Linux (WSL)¶

Windows Subsystem for Linux (WSL) is a platform provided by Microsoft onto which a Linux distribution (i.e. Ubuntu) can be installed. It provides a Linux shell within Windows that contains a functional Linux operating system, and thus can compile and run programs meant for Linux within that environment.

To install WSL on your machine, follow Microsoft’s provided installation instructions:

wsl --install

Now, type Ubuntu (or whatever other distribution you chose) into the Start Menu:

the Windows Start menu with ``Ubuntu`` in the search box; the top result is ``Ubuntu 20.04 LTS``

This will boot the operating system and start the installation within WSL, after which it will prompt you to set a username and password.

After installation completes, you now have a functional Linux operating system that you can access in the same way.

You have access to many Windows utilities from within the Linux shell, and many WSL utilities from the Windows shell. By default, the Windows partition is mounted at /mnt/c.

Microsoft provides documentation on interoperability between Windows and WSL.

If you have issues with stenv, please create a new GitHub issue or contact one of the following help desks:

HST Help Desk: https://stsci.service-now.com/hst
JWST Help Desk: https://stsci.service-now.com/jwst

Space Telescope Environment (`stenv`)¶

Getting Started¶

Conda Basics¶

Installation¶

Install Conda¶

Choose an `stenv` release¶

Activating an environment¶

Deleting an environment¶

Frequently Asked Questions¶

`stenv` doesn’t build on my system; what do I do?¶

Why isn’t _____ package in `stenv`?¶

What about Astroconda?¶

Developer Notes¶

`environment.yaml`¶

Adding a package to `stenv`¶

Pipeline Releases¶

Installation¶

File URLs¶

HST Data Processing (HSTDP)¶

Continuous Integration¶

Windows Support¶

Windows Subsystem for Linux (WSL)¶

Space Telescope Environment (stenv)¶

Getting Started¶

Conda Basics¶

Installation¶

Install Conda¶

Choose an stenv release¶

Activating an environment¶

Deleting an environment¶

Frequently Asked Questions¶

stenv doesn’t build on my system; what do I do?¶

Why isn’t _____ package in stenv?¶

What about Astroconda?¶

Developer Notes¶

environment.yaml¶

Adding a package to stenv¶

Pipeline Releases¶

Installation¶

File URLs¶

HST Data Processing (HSTDP)¶

Continuous Integration¶

Windows Support¶

Windows Subsystem for Linux (WSL)¶

Space Telescope Environment (`stenv`)¶

Choose an `stenv` release¶

`stenv` doesn’t build on my system; what do I do?¶

Why isn’t _____ package in `stenv`?¶

`environment.yaml`¶

Adding a package to `stenv`¶