I'm trying to create a 2-layer neural network, for that I first initialize weights and biases to random floats between 0 an 1 using numpy.random.rand. However, for some reason this process produces floats bigger than 1 for W1 (weight 1) whereas it works correctly for all other weights an biases. I can't understand why this happens, I thought maybe something affects the function from outside the function where I initialized the parameters, but I couldn't detect any part in the function that could be affected from outside the function.

import numpy as np

    ### CONSTANTS DEFINING THE MODEL ####
n_x = 12288     # num_px * num_px * 3
n_h = 7
n_y = 1
layers_dims = (n_x, n_h, n_y)

def initialize_parameters_deep(layer_dims):
    """
    Arguments:
    layer_dims -- python array (list) containing the dimensions of each layer in our network
    
    Returns:
    parameters -- python dictionary containing your parameters "W1", "b1","W2", "b2":
    """
    np.random.seed(1)
    parameters = {}

    parameters["W1"] = np.random.rand(n_h, n_x)  #(7, 12288)
    parameters["b1"] = np.random.rand(n_h, 1)    #(7)
    parameters["W2"] = np.random.rand(n_y, n_h)  #(7, 1)
    parameters["b2"] = np.random.rand(n_y, 1)    #(1)
    
    return parameters

parameters = initialize_parameters_deep(layers_dims)
print(parameters)

Output:

{'W1': array([[4.17022005e-01, 7.20324493e-01, 1.14374817e-04, ...,
        3.37562919e-01, 1.12292153e-01, 5.37047221e-01],
       [7.07934286e-01, 3.37726007e-01, 7.07954162e-01, ...,
        4.22040811e-01, 7.78593215e-01, 3.49866021e-01],
       [9.01338451e-01, 7.95132845e-03, 1.03777034e-01, ...,
        2.78602449e-01, 5.05813021e-02, 8.26828833e-01],
       ...,
       [5.62717083e-03, 6.58208224e-01, 3.88407263e-01, ...,
        5.56312618e-01, 8.69650932e-01, 1.00112287e-01],
       [4.16278934e-01, 4.56060621e-01, 9.33378848e-01, ...,
        9.52798385e-01, 9.41894584e-01, 4.44342962e-01],
       [8.89254832e-01, 6.42558949e-01, 2.29427262e-01, ...,
        8.05884494e-01, 1.80676088e-01, 6.12694420e-01]]), 'b1': array([[0.11933315],
       [0.50073416],
       [0.21336813],
       [0.14223935],
       [0.60809243],
       [0.41994954],
       [0.43137737]]), 'W2': array([[0.81360697, 0.44638382, 0.41794085, 0.08649817, 0.29957473,
        0.33706742, 0.24721952]]), 'b2': array([[0.92363097]])}

EDIT: Thanks for the help, didn't know about that e-representation.

CodePudding user response：

It's not generating floats bigger than 1, it's just representing them differently.

4.17022005e-01 is the same as 0.417022005, and 1.14374817e-04 is the same as 0.000114374817.

See here or here.

CodePudding user response：

The e-01, e-02, e-03, etc at the end of the W1 numbers just mean that the numbers are written in exponential format. So if you have for example 2.786e-01 that is the same as if it was written like (2.786/10) and that is the same as 0.2786. Same thing goes for: 2.786e-03 == (2.786/1000) == 0.002786. e 2 is 10^2 and e-2 is 1/(10^2).

CodePudding user response：

Pay attention to the final few characters printed when you print your weights parameter tensor, which gives e.g. e-01. This represents base-10 exponentiation, i.e. meaning that the value of a given weight is the number printed times 10 to the given power.

All of the powers are negative, meaning the weights have small but positive values in the range [0, 1].

For example, 4.17022005e-01 equals 0.417022005.