Mathematics • Machine Learning • Data Science
Sigmoid Function vs Logarithm: A Complete Mathematical and Practical Guide
The sigmoid function and the logarithm function are two fundamental mathematical functions that appear throughout mathematics, statistics, data science, machine learning, optimization, probability, information theory and scientific computing. Although they may look completely different, they are connected through exponential functions, inverse relationships and the logit transformation.
1. Introduction
Mathematical functions are rules that transform inputs into outputs. Some functions grow rapidly, some grow slowly, some oscillate, and some compress values into a particular interval. The sigmoid and logarithm belong to two very different categories of behavior, yet both are extremely important in modern data science.
If you are learning machine learning, statistics or artificial intelligence, you will repeatedly encounter expressions involving \(e^x\), \(\ln(x)\), probabilities, logits and sigmoid values. Understanding these concepts mathematically is much more useful than simply memorizing formulas.
The sigmoid function is especially important because it converts an unrestricted real-valued score into a number between zero and one. This makes it natural for representing probabilities in binary classification.
The natural logarithm performs almost the opposite conceptual operation. Instead of compressing arbitrary real numbers into a probability-like interval, it takes positive numbers and tells us which exponent of \(e\) produces them.
2. What Is the Sigmoid Function?
The standard sigmoid function, also called the logistic sigmoid, is defined by:
\[ \sigma(x) = \frac{1}{1 + e^{-x}} \]
The Greek letter sigma, \(\sigma\), is commonly used to represent this function. The input \(x\) can be any real number. That means \(x\) can be negative, zero, positive, very large or very small.
The remarkable property of the function is that regardless of how large or small the input becomes, the output remains strictly between zero and one.
For example, if \(x = 0\):
\[ \sigma(0) = \frac{1}{1+e^0} = \frac{1}{1+1} = \frac{1}{2} = 0.5 \]
If \(x\) is strongly positive, \(e^{-x}\) becomes very small. Therefore the denominator approaches one and the sigmoid approaches one.
If \(x\) is strongly negative, \(-x\) becomes strongly positive. Then \(e^{-x}\) becomes very large and the fraction approaches zero.
3. Understanding the Sigmoid Function Intuitively
Imagine that a machine learning model calculates a raw score. That score might be -10, -2.5, 0, 1.7, 4 or 100. A raw score is not automatically a probability. It can be any real number.
The sigmoid function provides a smooth conversion from that raw score into a probability-like value.
Consider these approximate values:
- \(\sigma(-5) \approx 0.0067\)
- \(\sigma(-2) \approx 0.1192\)
- \(\sigma(0) = 0.5\)
- \(\sigma(2) \approx 0.8808\)
- \(\sigma(5) \approx 0.9933\)
Notice the important pattern. A large negative score produces a value close to zero. A score of zero produces exactly 0.5. A large positive score produces a value close to one.
This does not mean that the sigmoid itself magically discovers probabilities. Rather, a model can be designed so that its output score is transformed by the sigmoid and interpreted as a probability under the assumptions of that model.
Click to explore the intuition
Think of \(x\) as evidence. Negative evidence pushes the output toward zero. Positive evidence pushes the output toward one. Around zero, the model is uncertain and small changes in the input can noticeably change the output.
This smooth behavior is useful because machine learning optimization generally works better with differentiable functions than with abrupt step functions.
4. Sigmoid Domain and Range
Domain
The domain of a function describes all valid input values. For the sigmoid:
\[ \text{Domain} = (-\infty,\infty) \]
There is no real number that causes the standard sigmoid formula to become undefined. The exponential \(e^{-x}\) exists for every real \(x\).
Range
The range describes all possible output values. For the sigmoid:
\[ 0 < \sigma(x) < 1 \]
The function never actually reaches zero or one for finite values of \(x\). Instead, it approaches those values asymptotically.
Mathematically:
\[ \lim_{x\to-\infty}\sigma(x)=0 \]
and:
\[ \lim_{x\to\infty}\sigma(x)=1 \]
5. Understanding the Sigmoid Graph
The graph of the standard sigmoid function has an S shape. This is one of its most recognizable characteristics.
The curve has three broad regions:
- A lower saturation region where the output is close to zero.
- A central transition region where the function changes rapidly.
- An upper saturation region where the output is close to one.
The central point occurs at \(x=0\), where the output equals 0.5.
The function is also symmetric around the point \((0,0.5)\) in the sense that:
\[ \sigma(-x)=1-\sigma(x) \]
For example, if \(\sigma(2)\approx0.8808\), then \(\sigma(-2)\approx0.1192\), and those values add to one.
Why does the curve flatten?
When \(x\) becomes very positive, \(e^{-x}\) approaches zero. The denominator therefore approaches one, so additional increases in \(x\) produce smaller and smaller changes in the output.
When \(x\) becomes very negative, \(e^{-x}\) becomes extremely large. Increasing the magnitude of the negative input further produces increasingly small changes in the final fraction.
6. Important Points on the Sigmoid Curve
Several points help develop intuition:
- \(x=-5\): output is approximately 0.0067.
- \(x=-2\): output is approximately 0.1192.
- \(x=-1\): output is approximately 0.2689.
- \(x=0\): output is exactly 0.5.
- \(x=1\): output is approximately 0.7311.
- \(x=2\): output is approximately 0.8808.
- \(x=5\): output is approximately 0.9933.
The point \(x=0\) is particularly important because it is also the point where the derivative reaches its maximum value.
Since:
\[ \sigma(0)=0.5 \]
and:
\[ \sigma'(0)=0.5(1-0.5)=0.25 \]
the maximum slope of the standard sigmoid is \(0.25\).
7. What Is the Natural Logarithm?
The natural logarithm is written as \(\ln(x)\). It is the logarithm whose base is Euler's number \(e\), where:
\[ e \approx 2.718281828459045 \]
The logarithm answers an exponent question.
If:
\[ e^y=x \]
then:
\[ \ln(x)=y \]
For example:
\[ \ln(e^3)=3 \]
because \(e\) raised to the third power is \(e^3\).
Another example is:
\[ \ln(1)=0 \]
because:
\[ e^0=1 \]
8. Understanding Logarithms Intuitively
Suppose someone tells you that \(e^x=20\). Instead of solving for \(x\) by repeatedly guessing, logarithms provide a direct mathematical notation:
\[ x=\ln(20) \]
The logarithm is therefore closely connected to exponential growth.
Another important property is that logarithms turn multiplication into addition:
\[ \ln(ab)=\ln(a)+\ln(b) \]
This property is one reason logarithms are so valuable in probability, statistics and information theory.
Similarly:
\[ \ln\left(\frac{a}{b}\right)=\ln(a)-\ln(b) \]
and:
\[ \ln(a^b)=b\ln(a) \]
Why is this useful in data science?
Multiplying many probabilities can create extremely small numbers. Taking logarithms converts multiplication into addition, making calculations easier to manage and often numerically more stable.
For example, instead of multiplying many probabilities: \(p_1p_2p_3\cdots p_n\), we can work with: \[ \ln(p_1)+\ln(p_2)+\cdots+\ln(p_n) \] and optimize the resulting log-likelihood.
9. Logarithm Domain and Range
Domain
The natural logarithm is defined only when:
\[ x>0 \]
Therefore:
\[ \text{Domain}=(0,\infty) \]
In the real number system, \(\ln(0)\) is undefined and \(\ln(x)\) is not a real number for negative \(x\).
Range
Although the input must be positive, the output can be any real number:
\[ \text{Range}=(-\infty,\infty) \]
As \(x\) approaches zero from the positive side:
\[ \lim_{x\to0^+}\ln(x)=-\infty \]
As \(x\) approaches positive infinity:
\[ \lim_{x\to\infty}\ln(x)=\infty \]
10. Understanding the Logarithm Graph
The natural logarithm graph is an increasing curve with a vertical asymptote at \(x=0\). It grows quickly near zero and then becomes progressively flatter.
Important points include:
- \(\ln(1)=0\)
- \(\ln(e)=1\)
- \(\ln(e^2)=2\)
- \(\ln(e^3)=3\)
The curve continues upward forever, but it does so increasingly slowly.
This is an important contrast with exponential growth. Exponential functions can grow extremely rapidly, while logarithms grow slowly.
11. The Connection Between Logarithms and Exponentials
The natural logarithm and exponential function are inverse functions.
If:
\[ y=e^x \]
then:
\[ x=\ln(y) \]
This gives the identities:
\[ \ln(e^x)=x \]
and:
\[ e^{\ln(x)}=x \qquad x>0 \]
These identities are fundamental to understanding both logarithms and sigmoid functions because the sigmoid itself contains an exponential term.
12. Sigmoid vs Logarithm
| Property | Sigmoid | Natural Logarithm |
|---|---|---|
| Formula | \(\sigma(x)=1/(1+e^{-x})\) | \(\ln(x)\) |
| Domain | \((-\infty,\infty)\) | \((0,\infty)\) |
| Range | \((0,1)\) | \((-\infty,\infty)\) |
| Graph | S-shaped | Increasing concave-down curve |
| Growth behavior | Saturates | Grows without bound, but slowly |
| Inverse | Logit | Exponential |
| Common use | Probability modeling and classification | Growth, likelihood, information and transformations |
The biggest conceptual difference is the direction of transformation. Sigmoid takes arbitrary real values and compresses them into a bounded interval. Logarithm takes positive values and maps them onto the entire real number line.
13. Derivatives and Rates of Change
A derivative describes how quickly a function changes with respect to its input. Derivatives are essential in calculus, optimization and machine learning.
The derivative of the natural logarithm is:
\[ \frac{d}{dx}\ln(x)=\frac{1}{x} \]
The derivative of the sigmoid is:
\[ \sigma'(x)=\sigma(x)(1-\sigma(x)) \]
Both derivatives tell us something about the shape of their respective graphs.
For the logarithm, \(1/x\) becomes smaller as \(x\) grows. This explains why the logarithm becomes flatter for large inputs.
For sigmoid, the derivative is largest around \(x=0\) and becomes very small toward both extremes.
14. Deriving the Sigmoid Derivative
Start with:
\[ \sigma(x)=\frac{1}{1+e^{-x}} \]
Rewrite it as:
\[ \sigma(x)=(1+e^{-x})^{-1} \]
Apply the chain rule:
\[ \sigma'(x)=-(1+e^{-x})^{-2}(-e^{-x}) \]
Therefore:
\[ \sigma'(x)=\frac{e^{-x}}{(1+e^{-x})^2} \]
Now observe that:
\[ \sigma(x)=\frac{1}{1+e^{-x}} \]
and:
\[ 1-\sigma(x) = 1-\frac{1}{1+e^{-x}} = \frac{e^{-x}}{1+e^{-x}} \]
Multiplying the two expressions gives:
\[ \sigma(x)(1-\sigma(x)) = \frac{e^{-x}}{(1+e^{-x})^2} \]
Therefore:
\[ \boxed{\sigma'(x)=\sigma(x)(1-\sigma(x))} \]
15. Deriving the Logarithm Derivative
The natural logarithm is the inverse of the exponential function. Let:
\[ y=\ln(x) \]
Then:
\[ x=e^y \]
Differentiate both sides with respect to \(x\):
\[ 1=e^y\frac{dy}{dx} \]
Since \(e^y=x\):
\[ 1=x\frac{dy}{dx} \]
Therefore:
\[ \boxed{\frac{dy}{dx}=\frac{1}{x}} \]
This derivative is positive for all valid inputs because \(x>0\). Therefore the natural logarithm is always increasing.
16. Inverse Functions
An inverse function reverses the transformation performed by another function. The exponential function reverses the natural logarithm:
\[ y=\ln(x) \quad\Longleftrightarrow\quad x=e^y \]
The sigmoid's inverse is the logit function:
\[ \operatorname{logit}(y)=\ln\left(\frac{y}{1-y}\right) \]
The logit takes a value strictly between zero and one and returns a real number.
For example, if \(y=0.5\):
\[ \operatorname{logit}(0.5) = \ln\left(\frac{0.5}{0.5}\right) = \ln(1) = 0 \]
17. Understanding the Logit Function
The logit function is extremely important because it connects probability space and unrestricted real-valued score space.
Start with:
\[ p=\frac{1}{1+e^{-x}} \]
Rearranging:
\[ p(1+e^{-x})=1 \]
\[ pe^{-x}=1-p \]
Therefore:
\[ e^{-x}=\frac{1-p}{p} \]
Taking the natural logarithm:
\[ -x=\ln\left(\frac{1-p}{p}\right) \]
Hence:
\[ x=\ln\left(\frac{p}{1-p}\right) \]
The quantity \(p/(1-p)\) is called the odds. Therefore the logit is also the logarithm of the odds:
\[ \operatorname{logit}(p)=\ln(\text{odds}) \]
18. Sigmoid and Probability
A probability must lie between zero and one. This immediately explains why sigmoid is useful in binary classification.
Suppose a model produces a score \(z\). The sigmoid converts it to:
\[ p=\sigma(z) \]
If \(z=0\), then \(p=0.5\). The model is exactly at the midpoint.
If \(z\) is positive, \(p>0.5\).
If \(z\) is negative, \(p<0.5\).
A common classification rule is to classify an observation as class 1 when \(p\ge0.5\), although the threshold can be changed depending on the problem.
For example, medical screening, fraud detection and spam detection may use different thresholds depending on the relative costs of false positives and false negatives.
19. Sigmoid in Machine Learning
Sigmoid has a long history in machine learning. It became especially well known through logistic regression and neural network activation functions.
In binary classification, a model may first compute a linear score:
\[ z=w_1x_1+w_2x_2+\cdots+w_nx_n+b \]
The sigmoid is then applied:
\[ p=\sigma(z) \]
Here, \(w_i\) are learned weights, \(x_i\) are input features and \(b\) is a bias term.
The sigmoid therefore sits between the unrestricted model score and the probability-like output.
Why not simply use the raw score as probability?
A raw linear score can be smaller than zero or larger than one. For example, a score of 4 cannot be interpreted directly as a probability because probabilities are restricted to the interval from zero to one.
Sigmoid provides a smooth transformation that respects that boundary.
20. Sigmoid in Logistic Regression
Logistic regression models the probability of a binary outcome. The model can be written as:
\[ p(y=1|x)=\sigma(w^Tx+b) \]
An equivalent interpretation is that the log-odds are linear:
\[ \ln\left(\frac{p}{1-p}\right)=w^Tx+b \]
This equation is one of the most important connections between sigmoid and logarithm.
The model is not merely "using sigmoid because it gives numbers between zero and one." There is a deeper statistical relationship between the linear predictor and the logarithm of the odds.
21. Why Logarithms Appear in Classification Loss
Logarithms are central to maximum likelihood estimation and classification loss. For binary classification, the log-loss is commonly written:
\[ L=-[y\ln(p)+(1-y)\ln(1-p)] \]
Here, \(y\) is the actual binary label and \(p\) is the predicted probability.
If \(y=1\), the expression becomes:
\[ L=-\ln(p) \]
Therefore predicting a probability close to one for a true class-1 observation produces a small loss.
But predicting a probability close to zero for a true class-1 observation produces a very large loss because:
\[ \lim_{p\to0^+}-\ln(p)=\infty \]
This creates a strong penalty for extremely confident incorrect predictions.
Why logarithm instead of a simple difference?
Logarithmic loss has a useful probabilistic interpretation through likelihood. It also converts products of probabilities into sums, which makes optimization and mathematical analysis much more convenient.
22. Logarithms and Information Theory
Logarithms are fundamental to information theory. A common definition of information content is:
\[ I(x)=-\log_2(p(x)) \]
The choice of logarithm base determines the unit. Base 2 produces bits, while natural logarithms are associated with nats.
Rare events carry more information because their probabilities are smaller. The negative logarithm converts small probabilities into larger information values.
This same mathematical structure appears in entropy, cross-entropy, Kullback-Leibler divergence and many machine learning objectives.
23. Numerical Behavior and Stability
Mathematical formulas can behave differently when implemented on a computer. This matters because computers have finite precision.
A naive sigmoid implementation:
def sigmoid(x): return 1 / (1 + math.exp(-x))can encounter overflow for very large negative values because \(-x\) becomes very large and the exponential may exceed the numerical range of the system.
A more stable implementation treats positive and negative values differently.
import math def stable_sigmoid(x): if x >= 0: z = math.exp(-x) return 1 / (1 + z) else: z = math.exp(x) return z / (1 + z)This avoids unnecessarily calculating a huge exponential.
24. Worked Mathematical Examples
Example 1: Calculate sigmoid at zero
\[ \sigma(0)=\frac{1}{1+e^0} \]
Since \(e^0=1\):
\[ \sigma(0)=\frac12=0.5 \]
Example 2: Calculate sigmoid at 2
\[ \sigma(2)=\frac{1}{1+e^{-2}} \]
Since \(e^{-2}\approx0.1353\):
\[ \sigma(2)\approx\frac{1}{1.1353}\approx0.8808 \]
Example 3: Calculate \(\ln(e^4)\)
Using the inverse relationship:
\[ \ln(e^4)=4 \]
Example 4: Calculate \(\ln(1)\)
Since \(e^0=1\):
\[ \ln(1)=0 \]
Example 5: Calculate the sigmoid derivative at zero
We know:
\[ \sigma'(x)=\sigma(x)(1-\sigma(x)) \]
At \(x=0\):
\[ \sigma'(0)=0.5(1-0.5)=0.25 \]
Example 6: Calculate a logit
Suppose \(p=0.8\).
\[ \operatorname{logit}(0.8) = \ln\left(\frac{0.8}{0.2}\right) = \ln(4) \approx1.3863 \]
Therefore a probability of 0.8 corresponds to approximately 1.3863 log-odds.
25. Code Examples
The following Python example calculates sigmoid, logarithm, logit and their derivatives. It is deliberately written using the standard library so that the mathematical relationships remain visible.
import math def sigmoid(x): return 1 / (1 + math.exp(-x)) def logit(p): return math.log(p / (1 - p)) def sigmoid_derivative(x): s = sigmoid(x) return s * (1 - s) def logarithm(x): return math.log(x) values = [-5, -2, -1, 0, 1, 2, 5] print("x\tSigmoid\t\tSigmoid Derivative") for x in values: print( f"{x}\t" f"{sigmoid(x):.6f}\t" f"{sigmoid_derivative(x):.6f}" ) print() print("ln(1) =", logarithm(1)) print("ln(e) =", logarithm(math.e)) print("ln(e^3) =", logarithm(math.exp(3))) print() print("logit(0.5) =", logit(0.5)) print("logit(0.8) =", logit(0.8))The important point is not merely that the program produces numerical output. The code directly reflects the mathematical definitions introduced earlier.
Breakdown of the code
math.exp(x)calculates \(e^x\).math.log(x)calculates the natural logarithm.sigmoid(x)implements \(1/(1+e^{-x})\).logit(p)implements \(\ln(p/(1-p))\).sigmoid_derivative(x)uses \(\sigma(x)(1-\sigma(x))\).
26. CLI Output Examples
The following is a representative command-line output produced by the Python example above.
x Sigmoid Sigmoid Derivative -5 0.006693 0.006648 -2 0.119203 0.104994 -1 0.268941 0.196612 0 0.500000 0.250000 1 0.731059 0.196612 2 0.880797 0.104994 5 0.993307 0.006648 ln(1) = 0.0 ln(e) = 1.0 ln(e^3) = 3.0 logit(0.5) = 0.0 logit(0.8) = 1.3862943611198908This output demonstrates several important properties at once. The sigmoid approaches zero for large negative values and approaches one for large positive values. Its derivative is largest at zero and becomes smaller toward the extremes.
The logarithm examples demonstrate its inverse relationship with the exponential function. The logit examples demonstrate the inverse relationship between sigmoid probability and log-odds.
python sigmoid_log_demo.py27. Interactive Learning Section
Use the controls below to experiment with sigmoid and logarithm values. Changing the input allows you to see how the functions behave without manually calculating every value.
Interactive Sigmoid Calculator
Result will appear here.
Interactive Natural Log Calculator
Result will appear here.
Interactive Logit Calculator
Result will appear here.
Try these values
- Sigmoid: -5, -2, 0, 2, 5.
- Logarithm: 0.1, 1, 2.71828, 10, 100.
- Logit: 0.01, 0.1, 0.5, 0.9, 0.99.
Notice how logit values become very negative as probability approaches zero and very positive as probability approaches one.
28. Common Mistakes
Mistake 1: Saying sigmoid can output exactly zero or one
For finite real inputs, standard sigmoid produces values strictly between zero and one. It approaches zero and one asymptotically.
Mistake 2: Treating every logarithm as natural logarithm
In mathematical contexts, \(\ln(x)\) specifically means the natural logarithm. The notation \(\log(x)\) can mean different bases depending on context. In many data science and programming environments, however, the default logarithm function is the natural logarithm.
Mistake 3: Calculating logarithm of zero
\(\ln(0)\) is undefined in the real number system. The function approaches negative infinity as the positive input approaches zero, but negative infinity is a limit, not an ordinary output at \(x=0\).
Mistake 4: Calculating real logarithm of a negative number
The real-valued natural logarithm requires \(x>0\). Complex logarithms can be defined for negative values, but that is a different mathematical setting.
Mistake 5: Confusing sigmoid with logit
Sigmoid maps real numbers to probabilities:
\[ \mathbb{R}\rightarrow(0,1) \]
Logit performs the reverse mapping:
\[ (0,1)\rightarrow\mathbb{R} \]
Mistake 6: Assuming sigmoid is always the best neural network activation
Sigmoid remains important, especially for binary probability outputs, but it is not universally the best hidden-layer activation. Modern neural networks often use alternatives such as ReLU-family activations because sigmoid can produce very small gradients in its saturation regions.
Mistake 7: Ignoring numerical stability
Directly evaluating exponentials for extremely large values can cause overflow or underflow. Production implementations should use numerically stable formulations where appropriate.
29. When Should You Use Each Function?
Use sigmoid when:
- You need a smooth mapping from real numbers to values between zero and one.
- You are modeling a binary outcome probability.
- You are working with logistic regression.
- You need the logistic transformation of a score.
- You want to convert log-odds into probability.
Use logarithms when:
- You need to solve exponential relationships.
- You need to convert multiplication into addition.
- You are working with likelihoods or log-likelihoods.
- You need a transformation that reduces the scale of positive values.
- You are working with information theory or entropy.
- You need the inverse of exponential growth.
In many machine learning systems, both functions appear together. Logistic regression is an excellent example: the model can be described in terms of log-odds using a logarithm and converted into probability using sigmoid.
30. Final Summary
The sigmoid function is:
\[ \sigma(x)=\frac{1}{1+e^{-x}} \]
It accepts every real number and returns a value strictly between zero and one. Its graph is S-shaped, it passes through \((0,0.5)\), and it approaches zero and one asymptotically.
Its derivative is:
\[ \sigma'(x)=\sigma(x)(1-\sigma(x)) \]
Its inverse is the logit:
\[ \operatorname{logit}(p) = \ln\left(\frac{p}{1-p}\right) \]
The natural logarithm is:
\[ \ln(x) \]
It answers the question: "What power of \(e\) produces \(x\)?" It is defined only for positive real numbers and has all real numbers as its range.
Its derivative is:
\[ \frac{d}{dx}\ln(x)=\frac{1}{x} \]
Its inverse is the exponential function:
\[ e^x \]
The deeper connection between the two functions becomes particularly clear in logistic regression. The sigmoid converts log-odds into probability, while the logit converts probability into log-odds.
- Sigmoid maps real numbers to values between 0 and 1.
- Natural logarithm maps positive numbers to all real numbers.
- Sigmoid is closely connected to the exponential function.
- Natural logarithm is the inverse of the exponential function.
- Logit is the inverse of sigmoid.
- Sigmoid is important in binary classification and probability modeling.
- Logarithms are essential in likelihood, information theory and exponential relationships.
- Both functions are differentiable on their respective domains.
- Understanding domain, range, graph, derivative and inverse gives a much deeper understanding than memorizing formulas.
31. Frequently Asked Questions
What is the sigmoid function?
The sigmoid function is \(\sigma(x)=1/(1+e^{-x})\). It maps every real input to a value strictly between zero and one.
Why is sigmoid useful for binary classification?
It converts an unrestricted real-valued model score into a value between zero and one, which can be interpreted as a probability under an appropriate statistical model.
What is the natural logarithm?
The natural logarithm \(\ln(x)\) is the logarithm with base \(e\). It tells us which exponent of \(e\) produces \(x\).
What is the domain of sigmoid?
The sigmoid function is defined for every real number, so its domain is \((-\infty,\infty)\).
What is the range of sigmoid?
Its range is \((0,1)\). The function approaches zero and one but does not reach either value for finite inputs.
What is the domain of ln(x)?
In the real number system, the domain of \(\ln(x)\) is \((0,\infty)\).
What is the derivative of sigmoid?
The derivative is \(\sigma(x)(1-\sigma(x))\).
What is the derivative of ln(x)?
The derivative is \(1/x\).
What is the inverse of sigmoid?
The inverse is the logit function: \(\ln(p/(1-p))\), defined for \(0
Why does logarithm appear in machine learning?
Logarithms are useful for likelihoods, log-loss, information theory, numerical transformations and converting multiplication into addition.
Are sigmoid and logarithm the same function?
No. They are fundamentally different functions. However, they are connected through exponentials, logits and probability transformations.
What happens to sigmoid when x approaches infinity?
The sigmoid approaches one: \(\lim_{x\to\infty}\sigma(x)=1\).
What happens to sigmoid when x approaches negative infinity?
The sigmoid approaches zero: \(\lim_{x\to-\infty}\sigma(x)=0\).
What happens to ln(x) when x approaches zero from the positive side?
It approaches negative infinity: \(\lim_{x\to0^+}\ln(x)=-\infty\).
Why is the sigmoid derivative small at extreme values?
At extreme values, sigmoid is close to zero or one. Since its derivative is \(\sigma(x)(1-\sigma(x))\), one of the two factors becomes very small.
What is the relationship between sigmoid and logit?
They are inverse functions. Sigmoid converts a real-valued log-odds score into a probability, while logit converts a probability into log-odds.
