Jensen-Shannon(JS) Divergence
The Jensen-Shannon distance measures the similarity between two probability distributions.
It is based on the Kullback-Leibler divergence but is a different, symmetric version of the Kullback-Leibler divergence measure. The Jensen-Shannon distance between two distributions, if they are similar, is 0.
It is defined as the square root of the Jensen-Shannon divergence, which is a measure of the average divergence of the two distributions from their mean.
The formula to compute Jensen-Shannon between P and Q is:
JS(P,Q) = sqrt( [KL(P,M) + KL(Q,M)] / 2 )
Where M is the average of P and Q i.e. M = (P + Q) / 2
Then, Jensen-Shannon is the square root of the average of KL(P,M) and KL(Q,M)
The Jensen-Shannon distance is often applied to compare the similarity of two probability distributions in a variety of fields, including information theory, protein surface comparison, bioinformatics, machine learning, and natural language processing. It is a popular choice because it is easy to calculate and has several valuable properties, such as being symmetric, bounded, and satisfying the triangle inequality.
JS can be used for both categorical and numerical features. The default threshold is 0.05 in AryaXAI
Liked the content? you'll love our emails!
Is Explainability critical for your 'AI' solutions?
Schedule a demo with our team to understand how AryaXAI can make your mission-critical 'AI' acceptable and aligned with all your stakeholders.