API
Artifacts
PkgUtility.deploy_artifact!
— FunctionWhat deploy_artifact!
function does are
- determine if the artifact already exists in the
art_toml
file - if
true
, skip the deployment - if
false
- copy the file(s) to
~/.julia/artifacts/ARTIFACT_SHA/
- compress the artifact file(s) to a
.tar.gz
file - calculate the hash value of the compressed
tar.gz
file - bind the artifact file to the
.toml
file
- copy the file(s) to
Method for this deployment is
deploy_artifact!(art_toml::String, art_name::String, art_locf::String, art_file::Vector{String}, art_tarf::String, art_urls::Vector{String}; new_file::Vector{String} = art_file)
Deploy the artifact, given
art_toml
Artifact.toml
file locationart_name
Artifact name identitfierart_locf
Local folder that stores the source filesart_file
Vector of the source file namesart_tarf
Folder location to store the compressed.tar.gz
fileart_urls
Vector of public urls, where the compressed files are to be uploaded (user need to upload the file manually)new_file
Optional. New file names of the copied files (same asart_file
by default)
Examples
# deploy art_1.txt and art_2.txt as test_art artifact
deploy_artifact!("Artifacts.toml", "test_art", "./", ["art_1.txt", "art_2.txt], "./", ["https://public.server.url"]);
# deploy art_1.txt and art_2.txt as test_art artifact with new names
deploy_artifact!("Artifacts.toml", "test_art", "./", ["art_1.txt", "art_2.txt], "./", ["https://public.server.url"]; new_files=["new_1.txt", "new_2.txt"]);
In many cases, one might want to copy all the files in a folder to the target artifact, and iterate the file names is not convenient at all. Thus, a readily usable method is provided for this purpose:
deploy_artifact!(art_toml::String, art_name::String, art_locf::String, art_tarf::String, art_urls::Vector{String})
Deploy the artifact, given
art_toml
Artifact.toml
file locationart_name
Artifact name identitfierart_locf
Local folder that stores the source files (all files will be copied into the artifact)art_tarf
Folder location to store the compressed.tar.gz
fileart_urls
Vector of public urls, where the compressed files are to be uploaded (user need to upload the file manually)
Examples
# deploy all files in target folder
deploy_artifact!("Artifacts.toml", "test_art", "./folder", "./", ["https://public.server.url"]);
DateTime
PkgUtility.parse_timestamp
— Functionparse_timestamp(timestamp::Union{Int,String}; in_format::String = "YYYYMMDD", out_format::String = "DOY")
parse_timestamp(year::Int, doy::Int; out_format::String = "DOY")
parse_timestamp(year::Int, doy::AbstractFloat; out_format::String = "DOY")
Convert timestamp, given
timestamp
Time stampin_format
Format oftimestamp
, default isYYYYMMDD
out_format
Output format, default isDOY
year
Year (in this case, the function will convert year and day to timestamp first)doy
Day of year (typically 1-365, 1-366 for leap years)
The input format (string or integer) supports YYYYMMDD
, YYYYMMDDhh
, YYYYMMDDhhmm
, and YYYYMMDDhhmmss
, where the labels are
YYYY
Year numberMM
Month numberDD
Day numberhh
Hour numbermm
Minute numberss
second number
The supported outputs are
DATE
ADates.Date
type variableDATETIME
ADates.DateTime
type variableDOY
A day of year integerFDOY
A day of year float
Examples
time = parse_timestamp(20200130; in_format="YYYYMMDD", out_format="FDOY");
time = parse_timestamp("20200130"; in_format="YYYYMMDD", out_format="FDOY");
time = parse_timestamp(2020, 100);
time = parse_timestamp(2020, 100.23435436);
PkgUtility.month_days
— Functionmonth_days(year::Int, month::Int)
Return the number of days per month, given
year
Yearmonth
Month
PkgUtility.month_ind
— Functionmonth_ind(year::Int, doy::Int)
month_ind(year::Int, doy::AbstractFloat)
Return the month index, given
year
Yeardoy
Day of year (typically 1-365, 1-366 for leap years)
PkgUtility.terror
— Functionterror(info::String)
Add a time tag to logging string, given
info
Infomation to display with @error
PkgUtility.tinfo
— Functiontinfo(info::String)
Add a time tag to logging string, given
info
Infomation to display with @info
PkgUtility.twarn
— Functiontwarn(info::String)
Add a time tag to logging string, given
info
Infomation to display with @warn
Display
PkgUtility.pretty_display!
— Functionpretty_display!(pvec::Union{Vector{Pair{String,String}}, Vector{Pair{String,Any}}, Vector{Pair{Any,String}}, Vector{Pair{Any,Any}}}, spaces::String = " ")
Display the pairs in a pretty way, given
pvec
Vector of pairs to displayspaces
Leading spaces before displaying the pair key
Examples
_pairs = ["A" => "b", "d" => "A", "rr" => ["ra" => "rB", "rD" => "ra"]];
pretty_display!(_pairs);
pretty_display!(_pairs, " ");
PkgUtility.send_email!
— Functionsend_email!(subject::String, from_email::String, to_email::String, body::String)
Send out email, given
subject
Email subjectfrom_email
The outgoing email addressto_email
Email address to send outbody
Main body of the email
Recursive tests
PkgUtility.FT_test
— FunctionFT_test(para::Array, FT)
FT_test(para::Number, FT)
FT_test(para::Union{Function, Module, String, Symbol}, FT)
FT_test(para::Any, FT)
Return true or false to determine if the FT is consistent, given
para
Parameter to run FT controlFT
Float type
If the testing variable is an array, the function will test if element type is float number:
- If true, the function tests if the element type is the same as given
FT
- If false, the function tests each element recursively
The variable to test maybe a struct, but FT_test
does not know the struct type name a priori. Thus, we try to read out the fields of the variable:
- If succeeds, the function test the fields recursively
- If fails, then do nothing
Example
struct SA
a
b
end
sa = SA(1, 2.0);
ft_1 = FT_test([1, 2, 3], Float64);
ft_2 = FT_test(Any[1, 1.0f0, 1.0e0], Float64);
ft_3 = FT_test([1, 2.0, "a"], Float64);
ft_4 = FT_test(sa, Float64);
PkgUtility.NaN_test
— FunctionLike FT_test
, same logic is used to test if all the elements within the tested variable are not NaN:
NaN_test(para::Array)
NaN_test(para::Number)
NaN_test(para::Union{Function, Module, String, Symbol})
NaN_test(para::Any)
Test if the variable is not NaN, given
para
Parameter to test
Example
struct SA
a
b
end
nan_1 = NaN_test(SA(1,2));
nan_2 = NaN_test(SA(1,NaN));
nan_3 = NaN_test([1,2,NaN]);
nan_4 = NaN_test([1,3,4]);
nan_5 = NaN_test([1,2,"a"]);
Numerical methods
PkgUtility.numerical∫
— Functionnumerical∫(f::Vector{FT}, Δx::Vector{FT}) where {FT<:AbstractFloat}
numerical∫(f::Vector{FT}, Δx::FT) where {FT<:AbstractFloat}
Return the intergal of given
f
f(x) for each xΔx
Δx for xnumerical∫(f::Function, xmin::FT, xmax::FT, n::Int) where {FT<:AbstractFloat} numerical∫(f::Function, xmin::FT, xmax::FT, xtol::FT = sqrt(eps(FT)), ytol::FT = sqrt(eps(FT))) where {FT<:AbstractFloat}
Return the integral of given
f
A functionx_min
Minimum limit of xx_max
Maximum limit of xn
Number of points in the x range (evenly stepped)x_tol
Tolerance of Δx (x/N)y_tol
Tolerance of the integral solution
PkgUtility.lower_quadratic
— Functionlower_quadratic(a::FT, b::FT, c::FT) where {FT<:AbstractFloat}
Return the lower quadratic solution or NaN, given
a
Parameter ina*x^2 + b*x + c = 0
b
Parameter ina*x^2 + b*x + c = 0
c
Parameter ina*x^2 + b*x + c = 0
PkgUtility.upper_quadratic
— Functionupper_quadratic(a::FT, b::FT, c::FT) where {FT<:AbstractFloat}
Return the upper quadratic solution or NaN, given
a
Parameter ina*x^2 + b*x + c = 0
b
Parameter ina*x^2 + b*x + c = 0
c
Parameter ina*x^2 + b*x + c = 0
Statistics
PkgUtility.nanmax
— Functionnanmax(x::Array)
Return the maximum of array ommiting the NaN, given
x
Array of numbers, can be NaN
PkgUtility.nanmean
— Functionnanmean(x::Array)
Return the mean of array by ommiting the NaN, given
x
Array of numbers, can be NaN
PkgUtility.nanmedian
— Functionnanmedian(x::Array)
Return the median of array by ommiting the NaN, given
x
Array of numbers, can be NaN
PkgUtility.nanmin
— Functionnanmin(x::Array)
Return the maximum of array ommiting the NaN, given
x
Array of numbers, can be NaN
PkgUtility.nanpercentile
— Functionnanpercentile(x::Array, p::Number)
Return the percentile by excluding the NaN of given
x
Array of datap
Percentile in[%]
PkgUtility.nanstd
— Functionnanstd(x::Array)
Return the std of array by ommiting the NaN, given
x
Array of numbers, can be NaN
```
PkgUtility.mae
— Functionmae(y::Array, pred::Array)
Return the mean absolute error by ommiting the NaN, given
y
Array of numbers, can be NaNpred
Array of predictions, can be NaN
PkgUtility.mape
— Functionmape(y::Array, pred::Array)
Return the mean absolute percentage error by ommiting the NaN, given
y
Array of numbers, can be NaNpred
Array of predictions, can be NaN
PkgUtility.mase
— Functionmase(y::Array, pred::Array)
Return the mean absolute standardized error by ommiting the NaN, given
y
Array of numbers, can be NaNpred
Array of predictions, can be NaN
PkgUtility.rmse
— Functionrmse(y::Array, pred::Array)
Return the root mean square error by ommiting the NaN, given
y
Array of numbers, can be NaNpred
Array of predictions, can be NaN