Questions tagged [sigmoid]
For questions about the sigmoid functions (in particular, the logistic functions) and the consequences of using them as activation functions in neural networks.
38 questions
2 votes
2 answers
131 views
When should I use a linear unit instead of sigmoid in the output layer?
In which types of learning tasks are linear units more useful than sigmoid activation functions in the output layer of a multi-layer neural network?
- 1,223
1 vote
3 answers
216 views
Why softmax/sigmoid use base e instead of 2?
Performing -ln(ε) in NumPy returns relatively small values like this: ...
2 votes
2 answers
184 views
How would you go from 1 to k hidden layers in Cybenko's result that neural networks are universal approximators?
Cybenko showed that if $\sigma$ is a sigmoidal, continuous function, then for any $\varepsilon > 0$, for any continuous function $f: [0, 1]^d \to \mathbb{R}$, there exists a function of the form $g:...
- 123
0 votes
0 answers
63 views
Find the relationship between data in this plot
Attached image. How would you find the relationship between independent variable (x) and dependent variable (y)? Is it linear or non-linear? What would the function looks like? P.S. I believe this is ...
2 votes
1 answer
108 views
If the output is 0.09, does this mean that the prediction is class 1 or 0?
I use a Keras EfficientNetB7 and transfer learning to solve a binary classification problem. I use tf.keras.layers.Dense(1, activation="sigmoid")(x) for ...
- 125
1 vote
1 answer
130 views
Approximate the chance of a number is prime using neural network
How to create a model that can give an output with a range of 0 to 1 with a sigmoid activation function where the value closer to 0 means the lesser chance that the input number is not prime and the ...
7 votes
4 answers
2k views
What does "e" do in the Sigmoid Activation Function?
Within the Sigmoid Squishification function, f(x) = 1/(1 + e^(-x)) "e" is unnecessary, as it can be replaced by any other value that is not ...
- 181
0 votes
1 answer
75 views
How can a Regression based Neural Network learn class thresholds?
I understand that to solve multilabel classification problems, we can use the softmax activation function in the output layer of the neural network. The softmax function outputs probabilities of each ...
- 43
0 votes
1 answer
129 views
Why is `SigmoidBinaryCrossEntropyLoss` in `DJL` implemented this way?
SigmoidBinaryCrossEntropyLoss implementation in DJL accepts two kinds of outputs from NNs: where sigmoid activation has already been applied. where raw NN output ...
- 1
0 votes
1 answer
146 views
How does a sigmoid neuron act like a perceptron in this scenario?
I have been reading Michael Nielsen’s book online on his website at http://neuralnetworksanddeeplearning.com/chap1.html. I am struggling to understand the second exercise: When c approaches infinity, ...
0 votes
2 answers
209 views
Is this the correct way to backpropagate a Neural Network?
I am writing a Neural Network frorm scratch. Below is what I have right now, based off of the math that I think I understand. ...
user55176
1 vote
1 answer
426 views
Does it make sense to provide a DQN with negative rewards for a network with relu and sigmoid activations?
The creation of negative rewards leads to the chance of Q-values being negative. However, networks with relu or sigmoid activations, just cannot predict negative values. This will lead to a case where ...
- 682
0 votes
1 answer
857 views
Do the values over 0.5 mean my model classified the data as a "1" label and vice versa?
I am doing binary classification using an LSTM and my output layer is 1 neuron with a sigmoid function. My labels are either 0 or 1. ...
- 3
0 votes
2 answers
2k views
Training a regression model on a set of values in 0-1 range to give 0-1 continual values
I have a textual dataset that has a set of real numbers as labels: L={0.0, 0.33, 0.5, 0.75, 1.0}, and I have a model that takes the text as input and has a Sigmoid output. If I train the model on this ...
- 123
3 votes
3 answers
6k views
Why is there tanh(x)*sigmoid(x) in a LSTM cell?
CONTEXT I was wondering why there are sigmoid and tanh activation functions in an LSTM cell. My intuition was based on the flow of tanh(x)*sigmoid(x) and the ...
- 242