Bidimensional Entropies

Functions for estimating the entropy of a two-dimensional univariate matrix.

While EntropyHub functions primarily apply to time series data, with the following bidimensional entropy functions one can estimate the entropy of two-dimensional (2D) matrices. Hence, bidimensional entropy functions are useful for applications such as image analysis.

IMPORTANT: Locked Matrix Size

Each bidimensional entropy function (SampEn2D, FuzzEn2D, DistEn2D, DispEn2D, EspEn2D, PermEn2D) has an important keyword argument - Lock. Bidimensional entropy functions are "locked" by default (Lock == true) to only permit matrices with a maximum size of 128 x 128.

The reason for this is because there are hundreds of millions of pairwise calculations performed in the estimation of bidimensional entropy, so memory errors often occur when storing data on RAM.

e.g. For a matrix of size [200 x 200], an embedding dimension (m) = 3, and a time delay (tau) = 1, there are 753,049,836 pairwise matrix comparisons (6,777,448,524 elemental subtractions). To pass matrices with sizes greater than [128 x 128], set Lock = false.

CAUTION: unlocking the permitted matrix size may cause your Julia IDE to crash.

EntropyHub._SampEn2D.SampEn2D — Function

SE2D, Phi1, Phi2 = SampEn2D(Mat)

Returns the bidimensional sample entropy estimate (SE2D) and the number of matched sub-matricess (m:Phi1, m+1:Phi2) estimated for the data matrix (Mat) using the default parameters: time delay = 1, radius distance threshold = 0.2*SD(Mat), logarithm = natural matrix template size = [floor(H/10) floor(W/10)], (where H and W represent the height (rows) and width (columns) of the data matrix Mat)

** The minimum dimension size of Mat must be > 10.**

SE2D, Phi1, Phi2 = SampEn2D(Mat::AbstractArray{T,2} where T<:Real; m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10),             
                                tau::Int=1, r::Real=0.2*std(Mat,corrected=false), Logx::Real=exp(1), Lock::Bool=true)

Returns the bidimensional sample entropy (SE2D) estimates for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

m - Template submatrix dimensions, an integer scalar (i.e. the same height and width) or a two-element vector of integers [height, width] with a minimum value > 1. (default: [floor(H/10) floor(W/10)])

tau - Time Delay, a positive integer (default: 1)

r - Distance Threshold, a positive scalar (default: 0.2*SD(Mat))

Logx - Logarithm base, a positive scalar (default: natural)

Lock - By default, SampEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, SampEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. (default: true)

      `WARNING: unlocking the permitted matrix size may cause your Julia
      IDE to crash.`

See also SampEn, FuzzEn2D, XSampEn, MSEn

References:

[1] Luiz Eduardo Virgili Silva, et al.,
     "Two-dimensional sample entropy: Assessing image texture 
     through irregularity." 
     Biomedical Physics & Engineering Express
     2.4 (2016): 045002.

source

EntropyHub._FuzzEn2D.FuzzEn2D — Function

Fuzz2D = FuzzEn2D(Mat)

Returns the bidimensional fuzzy entropy estimate (Fuzz2D) estimated for the data matrix (Mat) using the default parameters: time delay = 1, fuzzy function (Fx) = 'default', fuzzy function parameters (r) = [0.2, 2], logarithm = natural, template matrix size = [floor(H/10) floor(W/10)], (where H and W represent the height and width of the data matrix 'Mat')

** The minimum dimension size of Mat must be > 10.**

Fuzz2D = FuzzEn2D(Mat::AbstractArray{T,2} where T<:Real; m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10), 
                    tau::Int=1, r::Union{Real,Tuple{Real,Real}}=(.2*std(Mat, corrected=false),2), 
                        Fx::String="default", Logx::Real=exp(1), Lock::Bool=true)

Returns the bidimensional fuzzy entropy (Fuzz2D) estimates for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

tau - Time Delay, a positive integer (default: 1)

Fx - Fuzzy function name, one of the following: {"sigmoid", "modsampen", "default", "gudermannian", "bell", "triangular", "trapezoidal1", "trapezoidal2", "z_shaped", "gaussian", "constgaussian"}

r - Fuzzy function parameters, a 1 element scalar or a 2 element vector of positive values. The 'r' parameters for each fuzzy function are defined as follows:

      sigmoid:        r(1) = divisor of the exponential argument
                      r(2) = value subtracted from argument (pre-division)
      modsampen:      r(1) = divisor of the exponential argument
                      r(2) = value subtracted from argument (pre-division)
      default:        r (1) = divisor of the exponential argument
                      r(2) = argument exponent (pre-division)
      gudermannian:   r  = a scalar whose value is the numerator of
                           argument to gudermannian function:
                           GD(x) = atan(tanh(r/x))
      triangular:     r = a scalar whose value is the threshold (corner point) of the triangular function.
      trapezoidal1:   r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the trapezoid.
      trapezoidal2:   r(1) = a value corresponding to the upper corner point of the trapezoid.
                      r(2) = a value corresponding to the lower corner point of the trapezoid.
      z_shaped:       r = a scalar whose value corresponds to the upper (2r) and lower (r) corner points of the z-shape.
      bell:           r(1) = divisor of the distance value
                      r(2) = exponent of generalized bell-shaped function
      gaussian:       r = a scalar whose value scales the slope of the Gaussian curve.
      constgaussian:  r = a scalar whose value defines the lower threshod and shape of the Gaussian curve.                    
      [DEPRICATED] linear:       r  = an integer value. When r = 0, the
                        argument of the exponential function is 
                        normalised between [0 1]. When r = 1,
                        the minimuum value of the exponential 
                        argument is set to 0.

Logx - Logarithm base, a positive scalar (default: natural)

Lock - By default, FuzzEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, FuzzEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. (default: true)

      ` WARNING: unlocking the permitted matrix size may cause
      your Julia IDE to crash.`

See also SampEn2D, FuzzEn, XFuzzEn

References:

[1] Luiz Fernando Segato Dos Santos, et al.,
    "Multidimensional and fuzzy sample entropy (SampEnMF) for
    quantifying H&E histological images of colorectal cancer."
    Computers in biology and medicine 
    103 (2018): 148-160.

[2] Mirvana Hilal and Anne Humeau-Heurtier,
    "Bidimensional fuzzy entropy: Principle analysis and biomedical
    applications."
    41st Annual International Conference of the IEEE (EMBC) Society
    2019.

[3] Hamed Azami, et al.
    "Fuzzy Entropy Metrics for the Analysis of Biomedical Signals: 
    Assessment and Comparison"
    IEEE Access
    7 (2019): 104833-104847

source

EntropyHub._DistEn2D.DistEn2D — Function

Dist2D = DistEn2D(Mat)

Returns the bidimensional distribution entropy estimate (Dist2D) estimated for the data matrix (Mat) using the default parameters: time delay = 1, histogram binning method = "sturges", logarithm = natural, template matrix size = [floor(H/10) floor(W/10)], (where H and W represent the height (rows) and width (columns) of the data matrix Mat)

** The minimum number of rows and columns of Mat must be > 10.**

Dist2D = DistEn2D(Mat::AbstractArray{T,2} where T<:Real; m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10), tau::Int=1,
                    Bins::Union{Int,String}="Sturges", Logx::Real=2, Norm::Int=2, Lock::Bool=true)

Returns the bidimensional distribution entropy (Dist2D) estimate for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

m - Template submatrix dimensions, an integer scalar (i.e. the same height and width) or a two-element tuple of integers [height, width] with a minimum value > 1. [default: [floor(H/10) floor(W/10)]]

tau - Time Delay, a positive integer [default: 1]

Bins - Histogram bin selection method for distance distribution, an integer > 1 indicating the number of bins, or one of the following strings {"sturges", "sqrt", "rice", "doanes"`} [default: 'sturges']

Logx - Logarithm base, a positive scalar [default: natural]

      ** enter 0 for natural logarithm.**

Norm - Normalisation of Dist2D value, one of the following integers: [0] no normalisation. [1] normalises values of data matrix (Mat) to range [0 1]. [2] normalises values of data matrix (Mat) to range [0 1], and normalises the distribution entropy value (Dist2D) w.r.t the number of histogram bins. [default] [3] normalises the distribution entropy value w.r.t the number of histogram bins, without normalising data matrix values.

Lock - By default, DistEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, DistEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. [default: 'true'] WARNING: unlocking the permitted matrix size may cause your Julia IDE to crash.

See also DistEn, XDistEn, SampEn2D, FuzzEn2D, MSEn

References:

[1] Hamed Azami, Javier Escudero and Anne Humeau-Heurtier,
    "Bidimensional distribution entropy to analyze the irregularity
    of small-sized textures."
    IEEE Signal Processing Letters 
    24.9 (2017): 1338-1342.

source

EntropyHub._DispEn2D.DispEn2D — Function

Disp2D, RDE = DispEn2D(Mat)

Returns the bidimensional dispersion entropy estimate (Disp2D) and reverse bidimensional dispersion entropy (RDE) estimated for the data matrix (Mat) using the default parameters: time delay = 1, symbols = 3, logarithm = natural, data transform = normalised cumulative density function ('ncdf'), logarithm = natural, template matrix size = [floor(H/10) floor(W/10)], (where H and W represent the height (rows) and width (columns) of the data matrix Mat)

** The minimum number of rows and columns of Mat must be > 10.**

Disp2D, RDE = DispEn2D(Mat::AbstractArray{T,2} where T<:Real; 
                    m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10), tau::Int=1,
                        c::Int=3, Typex::String="ncdf", Logx::Real=exp(1), Norm::Bool=false, Lock::Bool=true)

Returns the bidimensional dispersion entropy (Disp2D) and reverse bidimensional distribution entropy (RDE) estimate for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

tau - Time Delay, a positive integer [default: 1]

c - Number of symbols, an integer > 1 Typex - Type of symbolic mapping transform, one of the following: {linear, kmeans, ncdf, equal} See the EntropyHub Guide for more info on these transforms. Logx - Logarithm base, a positive scalar [default: natural]

      ** enter 0 for natural logarithm.**

Norm - Normalisation of Disp2D value, a boolean: - [false] no normalisation - default - [true] normalises w.r.t number of possible dispersion patterns. Lock - By default, DispEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, DispEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. [default: 'true'] WARNING: unlocking the permitted matrix size may cause your Julia IDE to crash.

See also DispEn, DistEn2D, SampEn2D, FuzzEn2D, MSEn

References:

[1] Hamed Azami, et al.,
    "Two-dimensional dispersion entropy: An information-theoretic 
    method for irregularity analysis of images."
    Signal Processing: Image Communication, 
    75 (2019): 178-187.

source

EntropyHub._PermEn2D.PermEn2D — Function

Perm2D = PermEn2D(Mat)

Returns the bidimensional permutation entropy estimate (Perm2D) estimated for the data matrix (Mat) using the default parameters: time delay = 1, logarithm = natural, template matrix size = [floor(H/10) floor(W/10)], (where H and W represent the height (rows) and width (columns) of the data matrix Mat)

** The minimum dimension size of Mat must be > 10.**

Perm2D = PermEn2D(Mat::AbstractArray{T,2} where T<:Real; m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10),             
                                tau::Int=1, Norm::Bool=true, Logx::Real=exp(1), Lock::Bool=true)

Returns the bidimensional permutation entropy (Perm2D) estimates for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

tau - Time Delay, a positive integer (default: 1)

Norm - Normalization of permutation entropy estimate, a boolean (default: true)

Logx - Logarithm base, a positive scalar (default: natural)

Lock - By default, PermEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, SampEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. (default: true)

      `WARNING: unlocking the permitted matrix size may cause your Julia
      IDE to crash.`

NOTE - The original bidimensional permutation entropy algorithms [1][2] do not account for equal-valued elements of the embedding matrices. To overcome this, PermEn2D uses the lowest common rank for such instances. For example, given an embedding matrix A where, A = [3.4 5.5 7.3] |2.1 6 9.9| [7.3 1.1 2.1] would normally be mapped to an ordinal pattern like so, [3.4 5.5 7.3 2.1 6 9.9 7.3 1.1 2.1] => [ 8 4 9 1 2 5 3 7 6 ] However, indices 4 & 9, and 3 & 7 have the same values, 2.1 and 7.3 respectively. Instead, PermEn2D uses the ordinal pattern [ 8 4 4 1 2 5 3 3 6 ] where the lowest rank (4 & 3) are used instead (of 9 & 7). Therefore, the number of possible permutations is no longer (mxmy)!, but (mxmy)^(mxmy). Here, the PermEn2D value is normalized by the maximum Shannon entropy (Smax = log((mxmy)!) $assuming that no equal values are found in the permutation motif matrices$, as presented in [1].

See also SampEn2D, FuzzEn2D, DispEn2D, DistEn2D

References:

[1] Haroldo Ribeiro et al.,
        "Complexity-Entropy Causality Plane as a Complexity Measure 
        for Two-Dimensional Patterns"
        PLoS ONE (2012), 7(8):e40689, 

[2] Luciano Zunino and Haroldo Ribeiro,
        "Discriminating image textures with the multiscale
        two-dimensional complexity-entropy causality plane"
        Chaos, Solitons and Fractals,  91:679-688 (2016)

[3] Matthew Flood and Bernd Grimm,
        "EntropyHub: An Open-source Toolkit for Entropic Time Series Analysis"
        PLoS ONE (2021) 16(11): e0259448.

source

EntropyHub._EspEn2D.EspEn2D — Function

Esp2D,  = EspEn2D(Mat)

Returns the bidimensional Espinosa entropy estimate (Esp2D) estimated for the data matrix (Mat) using the default parameters: time delay = 1, tolerance threshold = 20, percentage similarity = 0.7 logarithm = natural, matrix template size = [floor(H/10) floor(W/10)], (where H and W represent the height (rows) and width (columns) of the data matrix Mat) ** The minimum number of rows and columns of Mat must be > 10.

Esp2D = EspEn2D(Mat::AbstractArray{T,2} where T<:Real; m::Union{Int,Tuple{Int,Int}}=floor.(Int, size(Mat)./10),             
                                tau::Int=1, r::Real=20, ps::Float=.7, Logx::Real=exp(1), Lock::Bool=true)

Returns the bidimensional Espinosa entropy (Esp2D) estimates for the data matrix (Mat) using the specified 'keyword' arguments:

Arguments:

tau - Time Delay, a positive integer (default: 1)

r - Tolerance threshold, a positive scalar (default: 20)

ps - Percentage similarity, a value in range [0 1], (default: 0.7)

Logx - Logarithm base, a positive scalar (default: natural)

Lock - By default, EspEn2D only permits matrices with a maximum size of 128 x 128 to prevent memory errors when storing data on RAM. e.g. For Mat = [200 x 200], m = 3, and tau = 1, EspEn2D creates a vector of 753049836 elements. To enable matrices greater than [128 x 128] elements, set Lock to false. (default: true)

      `WARNING: unlocking the permitted matrix size may cause your Julia
      IDE to crash.`

See also SampEn2D, FuzzEn2D, DispEn2D, DistEn2D, PermEn2D

References:

[1] Ricardo Espinosa, et al.,
    "Two-dimensional EspEn: A New Approach to Analyze Image Texture 
    by Irregularity." 
    Entropy,
    23:1261 (2021)

source