I'm new to pytorch and wrote a simple code as following to classify some inputs. The model input has 8*2 with batch size of 2 and the input layer in the model has 2 nodes. I don't know what is wrong!

X1=np.array([[2,1],[3,2],[-4,-1],[-1,-3],[2,-1],[3,-3],[-2,1],[-4,-2]])
Y1=np.array([0,0,0,0,1,1,1,1])
X=torch.tensor(X1)
Y=torch.tensor(Y1)

BATCH_SIZE=2
trainset= torch.utils.data.TensorDataset(X, Y)
trainloader = torch.utils.data.DataLoader(trainset, batch_size=BATCH_SIZE,
                                          shuffle=True, num_workers=1)  
from torch.nn.modules import flatten



learning_rate = 0.01
num_epochs = 20

device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
model = MyModel()
model = model.to(device)
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(model.parameters(), lr=learning_rate)


## compute accuracy
def get_accuracy(logit, target, batch_size):
    ''' Obtain accuracy for training round '''
    corrects = (torch.max(logit, 1)[1].view(target.size()).data == target.data).sum()
    accuracy = 100.0 * corrects/batch_size
    return accuracy.item()

model = MyModel()

# Commented out IPython magic to ensure Python compatibility.

for epoch in range(num_epochs):
    train_running_loss = 0.0
    train_acc = 0.0

    ## training step
    for inputs, labels in trainloader:

        #inputs=torch.flatten(inputs)
        inputs,labels=inputs.to(device), labels.to(device)
        #inputs = inputs.to(device)
        #labels = labels.to(device)

        optimizer.zero_grad()

        ## forward   backprop   loss
        
        print(inputs)
        
        outputs = model.forward(inputs)
        loss = criterion(outputs, labels)
        
        loss.backward()

        ## update model params
        optimizer.step()

        train_running_loss  = loss.detach().item()
        train_acc  = get_accuracy(outputs, labels, BATCH_SIZE)
    
        #model.train()
    model.eval()
    print('Epoch: %d | Loss: %.4f | Train Accuracy: %.2f'%(epoch, train_running_loss / i, train_acc/i))

And my model is as below:

class MyModel(nn.Module):
        def __init__(self):
            super(MyModel, self).__init__()
    
            self.d1 = nn.Linear(2,3)
            self.d2 = nn.Linear(3,1)
            self.init_weights()
    
        def init_weights(self):
            k1=torch.tensor([0.1,-0.72,0.94,-0.29,0.12,0.44])
            k1=torch.unsqueeze(torch.unsqueeze(k1,0),0)
            self.d1.weight.data=k1
            k2=torch.tensor([1,-1.16,-0.26])
            k2=torch.unsqueeze(torch.unsqueeze(k2,0),0)
            self.d2.weight.data=k2
    
        def forward(self, x):
            x = self.d1(x)
            x = F.tanh(x)
            x = self.d2(x)
            out = F.sigmoid(x)
    
            return out

Then I got an error:

---------------------------------------------------------------------------
RuntimeError                              Traceback (most recent call last)
<ipython-input-27-196d819d3ccd> in <module>
    101         print(inputs)
    102 
--> 103         outputs = model.forward(inputs)
    104         loss = criterion(outputs, labels)
    105 

2 frames
/usr/local/lib/python3.8/dist-packages/torch/nn/modules/linear.py in forward(self, input)
    112 
    113     def forward(self, input: Tensor) -> Tensor:
--> 114         return F.linear(input, self.weight, self.bias)
    115 
    116     def extra_repr(self) -> str:

RuntimeError: t() expects a tensor with <= 2 dimensions, but self is 3D

I flatten the input but nothing changed. What should I do to fix it?

CodePudding user response：

First of all, you don't need to invoke your model's forward pass by model.forward(x); using model(x) is good.

Second of all, what exactly are you trying to achieve via the init_weights method? You're unsqueezing k1 and k2 twice, giving them the shape of (1, 1, x) which is 3D which is what the error is telling you. torch.nn.Linear performs a matrix multiplication with a 2D matrix, so you can't use a 3D one. torch.nn.Linear already initializes the weights via Kaiming initialization [1] so I'm not sure what you're trying to achieve here.

Changing the init_weights method to:

def init_weights(self):
    k1 = torch.tensor([0.1, -0.72, 0.94, -0.29, 0.12, 0.44])
    k1 = k1.reshape(self.d1.weight.shape)
    self.d1.weight.data = k1
            
    k2 = torch.tensor([1, -1.16, -0.26])
    k2 = k2.reshape(self.d2.weight.shape)
    self.d2.weight.data = k2

and changing the type of inputs from Long to Float (i.e., model(inputs.float())) should solve your problem.

References
[1] https://github.com/pytorch/pytorch/blob/0dceaf07cd1236859953b6f85a61dc4411d10f87/torch/nn/modules/linear.py#L103