NetcdfIO.jl

Functions to read and write netcdf files. The core functions to read data from netcdf files is adapted from an old version of NCDatasets.jl. Some convenient features have been added, for example, one can read the dataset without diving into the nested structure of dataset/group/variable, with the use of find_variable function which detect the variable name recursively.

Installation

julia> using Pkg;
julia> Pkg.add("NetcdfIO");

Read the general information

NetcdfIO.read_dimnames — Function

read_dimnames(file::String)

Return all the names of the dimensions, given

file Path of the netcdf dataset

source

NetcdfIO.read_varnames — Function

read_varnames(ds::Dataset)
read_varnames(file::String)

Return all the names of the variables (excluding the dimensions), given

ds NCDatasets.Dataset type dataset
file Path of the netcdf dataset

source

NetcdfIO.read_dims — Function

read_dims(file::String, var_name::UnionNameTypes)

Return the dimensions and size of a NetCDF dataset, given

file Path of the netcdf dataset
var_name Variable name

source

NetcdfIO.read_attributes — Function

read_attributes(ds::Dataset) -> Dict{String, Any}
read_attributes(file::String) -> Dict{String, Any}
read_attributes(ds::Dataset, var_name::UnionNameTypes) -> Dict{String, Any}
read_attributes(file::String, var_name::UnionNameTypes) -> Dict{String, Any}

Return all the attributes of a NetCDF dataset or variable, given

ds NCDatasets.Dataset type dataset
var_name Variable name
file Path of the netcdf dataset

source

Read the variable

NetcdfIO.find_variable — Function

find_variable(ds::Dataset, var_name::UnionNameTypes)

Return the path to dataset if it exists, given

ds NCDatasets.Dataset type dataset
var_name Variable to read

source

NetcdfIO.read_nc — Function

read_nc(file::String, var_name::UnionNameTypes; transform::Bool = true)
read_nc(T, file::String, var_name::UnionNameTypes; transform::Bool = true)

Read entire data from NC file, given

file Path of the netcdf dataset
var_name Variable to read
transform If true, transform the data using NCDatasets rules, otherwise read the raw data
T Number type

read_nc(file::String, var_name::UnionNameTypes, indz::Int; transform::Bool = true)
read_nc(T, file::String, var_name::UnionNameTypes, indz::Int; transform::Bool = true)

Read a subset from nc file, given

file Path of the netcdf dataset
var_name Variable name
indz The 3rd index of subset data to read
transform If true, transform the data using NCDatasets rules, otherwise read the raw data
T Number type

Note that the dataset must be a 1D or 3D array to use this method.

read_nc(file::String, var_name::UnionNameTypes, indx::Int, indy::Int; transform::Bool = true)
read_nc(T, file::String, var_name::UnionNameTypes, indx::Int, indy::Int; transform::Bool = true)

Read the subset data for a grid, given

file Path of the netcdf dataset
var_name Variable name
indx The 1st index of subset data to read, typically longitude
indy The 2nd index of subset data to read, typically latitude
transform If true, transform the data using NCDatasets rules, otherwise read the raw data
T Number type

read_nc(file::String, var_name::UnionNameTypes, indx::Int, indy::Int, indz::Int; transform::Bool = true)
read_nc(T, file::String, var_name::UnionNameTypes, indx::Int, indy::Int, indz::Int; transform::Bool = true)

Read the data at a grid, given

file Path of the netcdf dataset
var_name Variable name
indx The 1st index of subset data to read, typically longitude
indy The 2nd index of subset data to read, typically latitude
indz The 3rd index of subset data to read, typically time
transform If true, transform the data using NCDatasets rules, otherwise read the raw data
T Number type

read_nc(file::String, selections::Vector{<:UnionNameTypes} = read_varnames(file); transform::Bool = true)

Read the selected variables from a netcdf file as a DataFrame, given

file Path of the netcdf dataset
selections Variables to read from the file
transform If true, transform the data using NCDatasets rules, otherwise read the raw data

read_nc(file::String, var_name::UnionNameTypes, dim_array::Vector; transform::Bool = true)

Read parts of the data specified in an array

file Path of the netcdf dataset
var_name Variable name
dim_array Vector containing the parts of the data to read
transform If true, transform the data using NCDatasets rules, otherwise read the raw data

source

Write to a new file

NetcdfIO.create_nc! — Function

create_nc!(file::String)

Create an empty netcdf file, given

file Path to save the netcdf dataset

create_nc!(file::String, dim_names::Vector{String}, dim_sizes::Vector)

Create an empty netcdf file with dimensions, given

file Path to save the netcdf dataset
dim_names Dimension names in the netcdf file
dim_sizes Sizes of the dimensions (must be Integer or Inf), the dimension is growable if size is Integer 0

source

NetcdfIO.add_nc_dim! — Function

add_nc_dim!(ds::Dataset, dim_name::String, dim_size::Int)
add_nc_dim!(ds::Dataset, dim_name::String, dim_size::AbstractFloat)
add_nc_dim!(file::String, dim_name::String, dim_size::Union{AbstractFloat,Int})

Add dimension information to netcdf dataset, given

ds A NCDatasets.Dataset type dataset
dim_name Dimension name
dim_size Integer dimension size (0 for Inf, growable)

source

NetcdfIO.append_nc! — Function

append_nc!(ds::Dataset,
           var_name::UnionNameTypes,
           var_data::Array{T,N},
           var_attributes::UnionAttrTypes,
           dim_names::Vector{String};
           deflatelevel::Union{Int,Nothing} = 4) where {T<:Union{AbstractFloat,Integer,String},N}
append_nc!(file::String,
           var_name::UnionNameTypes,
           var_data::Array{T,N},
           var_attributes::UnionAttrTypes,
           dim_names::Vector{String};
           args...) where {T<:Union{AbstractFloat,Integer,String},N}

Append data to existing netcdf dataset, given

ds A NCDatasets.Dataset type dataset
var_name New variable name to write to
var_data New variable data to write, can be integer, float, and string with N dimens
var_attributes New variable attributes
dim_names Dimension names in the netcdf file
deflatelevel Compression level fro NetCDF, default is 4
file Path of the netcdf dataset

source

Add new data to unlimited dim

NetcdfIO.grow_nc! — Function

grow_nc!(ds::Dataset, var_name::UnionNameTypes, new_data::Union{AbstractFloat,Array,Integer,String}, pending::Bool)
grow_nc!(file::String, var_name::UnionNameTypes, new_data::Union{AbstractFloat,Array,Integer,String}, pending::Bool)

Grow the netcdf dataset, given

ds A NCDatasets.Dataset type dataset
var_name New variable name to write to
new_data New data to grow, can be integer, float, and string with N dimens
pending If true, the new data is appened to the end (growth); if false, the data will replace the ones from the bottom (when dimension has already growed)
file Path of the netcdf dataset

Note that if there are more variables to grow at the same time, set pending to true only for the first time you call this function, and set pending to false for the rest variables.

source

Automatic attribute detection (for global data)

NetcdfIO.save_nc! — Function

save_nc!(file::String,
         var_name::UnionNameTypes,
         var_data::Array{T,N},
         var_attributes::Union{Dict{String,Any}, OrderedDict{String,Any}};
         var_dims::Vector{String} = N == 2 ? ["lon", "lat"] : ["lon", "lat", "ind"],
         deflatelevel::Union{Int,Nothing} = 4,
         growable::Bool = false) where {T<:Union{AbstractFloat,Integer,String},N}

Save the 1D, 2D, or 3D data as netcdf file, given

file Path to save the dataset
var_name Variable name for the data in the NC file
var_data Data to save
var_attributes Variable attributes for the data, such as unit and long name
var_dims Dimension name of each dimension of the variable data
deflatelevel Compression level fro NetCDF, default is 4
growable If true, make index growable, default is false

Note that this is a wrapper function of createnc and appendnc:

If var_data is 1D, the dim is set to ind
If vardata is 2D, and no vardims are given, the dims are set to lon and lat
If vardata is 3D, and no vardims are given, the dims are set to lon, lat, and ind

save_nc!(file::String,
         df::DataFrame,
         var_names::Vector{<:UnionNameTypes},
         var_attributes_vec::UnionAttrVecTypes;
         deflatelevel::Union{Int,Nothing} = 4,
         growable::Bool = false)
save_nc!(file::String, df::DataFrame; deflatelevel::Union{Int,Nothing} = 4, growable::Bool = false)

Save DataFrame to NetCDF, given

file Path to save the data
df DataFrame to save
var_names The label of data in DataFrame to save
var_attributes Variable attributes for the data to save
deflatelevel Compression level fro NetCDF, default is 4
growable If true, make index growable, default is false

source

NetcdfIO.detect_attribute — Function

detect_attribute(var_name::String, wavelength::Union{Int,Nothing} = nothing; showwarning::Bool = true)

Return an ordered dictionary of variable attributes, given

var_name Name of the variable
wavelength Wavelength in nm for wavelength-dependent variables like SIF, default is nothing
showwarning If true, show a warning when the variable name is not recognized, default is true

source

Support to HDF4

NetcdfIO.switch_netcdf_lib! — Function

switch_netcdf_lib!(; use_default::Bool = true, user_defined::String = "/home/runner/.julia/conda/3/lib/libnetcdf.so")

Switch between the default NetCDF library and a user-defined one, given

use_default Whether to use the default libnetcdf library shipped with NCDatasets.jl
user_defined The path to the user-defined libnetcdf library (used only when use_default is false)

source