API

Artifacts

PkgUtility.deploy_artifact!Function

What deploy_artifact! function does are

  • determine if the artifact already exists in the art_toml file
  • if true, skip the deployment
  • if false
    • copy the file(s) to ~/.julia/artifacts/ARTIFACT_SHA/
    • compress the artifact file(s) to a .tar.gz file
    • calculate the hash value of the compressed tar.gz file
    • bind the artifact file to the .toml file

Method for this deployment is

deploy_artifact!(art_toml::String, art_name::String, art_locf::String, art_file::Vector{String}, art_tarf::String, art_urls::Vector{String}; new_file::Vector{String} = art_file)

Deploy the artifact, given

  • art_toml Artifact .toml file location
  • art_name Artifact name identitfier
  • art_locf Local folder that stores the source files
  • art_file Vector of the source file names
  • art_tarf Folder location to store the compressed .tar.gz file
  • art_urls Vector of public urls, where the compressed files are to be uploaded (user need to upload the file manually)
  • new_file Optional. New file names of the copied files (same as art_file by default)

Examples

# deploy art_1.txt and art_2.txt as test_art artifact
deploy_artifact!("Artifacts.toml", "test_art", "./", ["art_1.txt", "art_2.txt], "./", ["https://public.server.url"]);

# deploy art_1.txt and art_2.txt as test_art artifact with new names
deploy_artifact!("Artifacts.toml", "test_art", "./", ["art_1.txt", "art_2.txt], "./", ["https://public.server.url"]; new_files=["new_1.txt", "new_2.txt"]);

In many cases, one might want to copy all the files in a folder to the target artifact, and iterate the file names is not convenient at all. Thus, a readily usable method is provided for this purpose:

deploy_artifact!(art_toml::String, art_name::String, art_locf::String, art_tarf::String, art_urls::Vector{String})

Deploy the artifact, given

  • art_toml Artifact .toml file location
  • art_name Artifact name identitfier
  • art_locf Local folder that stores the source files (all files will be copied into the artifact)
  • art_tarf Folder location to store the compressed .tar.gz file
  • art_urls Vector of public urls, where the compressed files are to be uploaded (user need to upload the file manually)

Examples

# deploy all files in target folder
deploy_artifact!("Artifacts.toml", "test_art", "./folder", "./", ["https://public.server.url"]);
source

DateTime

PkgUtility.parse_timestampFunction
parse_timestamp(timestamp::Union{Int,String}; in_format::String = "YYYYMMDD", out_format::String = "DOY")
parse_timestamp(year::Int, doy::Int; out_format::String = "DOY")
parse_timestamp(year::Int, doy::AbstractFloat; out_format::String = "DOY")

Convert timestamp, given

  • timestamp Time stamp
  • in_format Format of timestamp, default is YYYYMMDD
  • out_format Output format, default is DOY
  • year Year (in this case, the function will convert year and day to timestamp first)
  • doy Day of year (typically 1-365, 1-366 for leap years)

The input format (string or integer) supports YYYYMMDD, YYYYMMDDhh, YYYYMMDDhhmm, and YYYYMMDDhhmmss, where the labels are

  • YYYY Year number
  • MM Month number
  • DD Day number
  • hh Hour number
  • mm Minute number
  • ss second number

The supported outputs are

  • DATE A Dates.Date type variable
  • DATETIME A Dates.DateTime type variable
  • DOY A day of year integer
  • FDOY A day of year float

Examples

time = parse_timestamp(20200130; in_format="YYYYMMDD", out_format="FDOY");
time = parse_timestamp("20200130"; in_format="YYYYMMDD", out_format="FDOY");
time = parse_timestamp(2020, 100);
time = parse_timestamp(2020, 100.23435436);
source
PkgUtility.month_daysFunction
month_days(year::Int, month::Int)

Return the number of days per month, given

  • year Year
  • month Month
source
PkgUtility.month_indFunction
month_ind(year::Int, doy::Int)
month_ind(year::Int, doy::AbstractFloat)

Return the month index, given

  • year Year
  • doy Day of year (typically 1-365, 1-366 for leap years)
source
PkgUtility.terrorFunction
terror(info::String)

Add a time tag to logging string, given

  • info Infomation to display with @error
source
PkgUtility.tinfoFunction
tinfo(info::String)

Add a time tag to logging string, given

  • info Infomation to display with @info
source
PkgUtility.twarnFunction
twarn(info::String)

Add a time tag to logging string, given

  • info Infomation to display with @warn
source

Display

PkgUtility.pretty_display!Function
pretty_display!(pvec::Union{Vector{Pair{String,String}}, Vector{Pair{String,Any}}, Vector{Pair{Any,String}}, Vector{Pair{Any,Any}}}, spaces::String = "    ")

Display the pairs in a pretty way, given

  • pvec Vector of pairs to display
  • spaces Leading spaces before displaying the pair key

Examples

_pairs = ["A" => "b", "d" => "A", "rr" => ["ra" => "rB", "rD" => "ra"]];
pretty_display!(_pairs);
pretty_display!(_pairs, "  ");
source

Email

PkgUtility.send_email!Function
send_email!(subject::String, from_email::String, to_email::String, body::String)

Send out email, given

  • subject Email subject
  • from_email The outgoing email address
  • to_email Email address to send out
  • body Main body of the email
source

Recursive tests

PkgUtility.FT_testFunction
FT_test(para::Array, FT)
FT_test(para::Number, FT)
FT_test(para::Union{Function, Module, String, Symbol}, FT)
FT_test(para::Any, FT)

Return true or false to determine if the FT is consistent, given

  • para Parameter to run FT control
  • FT Float type

If the testing variable is an array, the function will test if element type is float number:

  • If true, the function tests if the element type is the same as given FT
  • If false, the function tests each element recursively

The variable to test maybe a struct, but FT_test does not know the struct type name a priori. Thus, we try to read out the fields of the variable:

  • If succeeds, the function test the fields recursively
  • If fails, then do nothing

Example

struct SA
    a
    b
end
sa = SA(1, 2.0);

ft_1 = FT_test([1, 2, 3], Float64);
ft_2 = FT_test(Any[1, 1.0f0, 1.0e0], Float64);
ft_3 = FT_test([1, 2.0, "a"], Float64);
ft_4 = FT_test(sa, Float64);
source
PkgUtility.NaN_testFunction

Like FT_test, same logic is used to test if all the elements within the tested variable are not NaN:

NaN_test(para::Array)
NaN_test(para::Number)
NaN_test(para::Union{Function, Module, String, Symbol})
NaN_test(para::Any)

Test if the variable is not NaN, given

  • para Parameter to test

Example

struct SA
    a
    b
end

nan_1 = NaN_test(SA(1,2));
nan_2 = NaN_test(SA(1,NaN));
nan_3 = NaN_test([1,2,NaN]);
nan_4 = NaN_test([1,3,4]);
nan_5 = NaN_test([1,2,"a"]);
source

Numerical methods

PkgUtility.numerical∫Function
numerical∫(f::Vector{FT}, Δx::Vector{FT}) where {FT<:AbstractFloat}
numerical∫(f::Vector{FT}, Δx::FT) where {FT<:AbstractFloat}

Return the intergal of given

  • f f(x) for each x

  • Δx Δx for x

    numerical∫(f::Function, xmin::FT, xmax::FT, n::Int) where {FT<:AbstractFloat} numerical∫(f::Function, xmin::FT, xmax::FT, xtol::FT = sqrt(eps(FT)), ytol::FT = sqrt(eps(FT))) where {FT<:AbstractFloat}

Return the integral of given

  • f A function
  • x_min Minimum limit of x
  • x_max Maximum limit of x
  • n Number of points in the x range (evenly stepped)
  • x_tol Tolerance of Δx (x/N)
  • y_tol Tolerance of the integral solution
source
PkgUtility.lower_quadraticFunction
lower_quadratic(a::FT, b::FT, c::FT) where {FT<:AbstractFloat}

Return the lower quadratic solution or NaN, given

  • a Parameter in a*x^2 + b*x + c = 0
  • b Parameter in a*x^2 + b*x + c = 0
  • c Parameter in a*x^2 + b*x + c = 0
source
PkgUtility.upper_quadraticFunction
upper_quadratic(a::FT, b::FT, c::FT) where {FT<:AbstractFloat}

Return the upper quadratic solution or NaN, given

  • a Parameter in a*x^2 + b*x + c = 0
  • b Parameter in a*x^2 + b*x + c = 0
  • c Parameter in a*x^2 + b*x + c = 0
source

Statistics

PkgUtility.nanmaxFunction
nanmax(x::Array)

Return the maximum of array ommiting the NaN, given

  • x Array of numbers, can be NaN
source
PkgUtility.nanmeanFunction
nanmean(x::Array)

Return the mean of array by ommiting the NaN, given

  • x Array of numbers, can be NaN
source
PkgUtility.nanmedianFunction
nanmedian(x::Array)

Return the median of array by ommiting the NaN, given

  • x Array of numbers, can be NaN
source
PkgUtility.nanminFunction
nanmin(x::Array)

Return the maximum of array ommiting the NaN, given

  • x Array of numbers, can be NaN
source
PkgUtility.nanpercentileFunction
nanpercentile(x::Array, p::Number)

Return the percentile by excluding the NaN of given

  • x Array of data
  • p Percentile in [%]
source
PkgUtility.nanstdFunction
nanstd(x::Array)

Return the std of array by ommiting the NaN, given

  • x Array of numbers, can be NaN

```

source
PkgUtility.maeFunction
mae(y::Array, pred::Array)

Return the mean absolute error by ommiting the NaN, given

  • y Array of numbers, can be NaN
  • pred Array of predictions, can be NaN
source
PkgUtility.mapeFunction
mape(y::Array, pred::Array)

Return the mean absolute percentage error by ommiting the NaN, given

  • y Array of numbers, can be NaN
  • pred Array of predictions, can be NaN
source
PkgUtility.maseFunction
mase(y::Array, pred::Array)

Return the mean absolute standardized error by ommiting the NaN, given

  • y Array of numbers, can be NaN
  • pred Array of predictions, can be NaN
source
PkgUtility.rmseFunction
rmse(y::Array, pred::Array)

Return the root mean square error by ommiting the NaN, given

  • y Array of numbers, can be NaN
  • pred Array of predictions, can be NaN
source