LeakyReLU

 LeakyReLU

https://pytorch.org/docs/stable/generated/torch.nn.LeakyReLU.html

$$\text{LeakyReLU}(x)=\max (0,x)+\text{negative\_slope}\ast \min (0,x)$$

$$or$$

$$\begin{array}{l}x,\end{array}$$if x≥0

$$\begin{array}{l}\text{ = negative\_slope}\times x,\end{array}$$

>>> m = nn.LeakyReLU(0.1)
>>> input = torch.randn(2)
>>> output = m(input)