Deep Learning | Thread Categories

Is building a deep learning framework from scratch actually worth it?

I’ve spent the last few months building neural network components from scratch to better understand how deep learning works. So far I’ve implemented dense layers, convolutional layers, activation functions, backpropagation, optimizers like SGD and Adam, and basic training loops without relying on PyTorch or TensorFlow. Here’s a simplified example of one of my training loops:(Read More)

I’ve spent the last few months building neural network components from scratch to better understand how deep learning works. So far I’ve implemented dense layers, convolutional layers, activation functions, backpropagation, optimizers like SGD and Adam, and basic training loops without relying on PyTorch or TensorFlow.

Here’s a simplified example of one of my training loops:

for epoch in range(epochs):
    predictions = model.forward(X_train)
    loss = loss_fn(predictions, y_train)

    gradients = loss_fn.backward()
    model.backward(gradients)

    optimizer.step()

The project has been a great learning experience, but I’m starting to wonder whether continuing to add more features is the best use of my time.

Would employers or researchers actually value a project like this, or is it better to shift focus toward building real-world applications with existing frameworks like PyTorch or TensorFlow?

I’m curious how others approached this stage in their learning journey.

0 3 60 2 weeks ago

Subscriber

Nicola

July 30, 2026

How do you debug a deep learning model that won’t improve?

I’m training a model, but the loss stops improving after a few epochs. I’ve tried changing the learning rate and training longer, but the results are still poor. What are the first things you check when a deep learning model doesn’t seem to learn? These are phrased to encourage discussion and experience-sharing, making them well(Read More)

I’m training a model, but the loss stops improving after a few epochs. I’ve tried changing the learning rate and training longer, but the results are still poor.

What are the first things you check when a deep learning model doesn’t seem to learn?

These are phrased to encourage discussion and experience-sharing, making them well suited for Reddit or a community forum.

0 1 50 2 weeks ago

Subscriber

Oscar

July 30, 2026

How can I prevent overfitting in a convolutional neural network while training on a small?

I’m training a CNN on a relatively small image dataset, and the training accuracy quickly reaches near 100%, but validation accuracy stagnates and then drops. I suspect overfitting is the issue. Here’s a simplified version of my training code in PyTorch: import torchimport torch.nn as nnimport torch.optim as optimfrom torchvision import datasets, transforms # Datasettrain_dataset(Read More)

I’m training a CNN on a relatively small image dataset, and the training accuracy quickly reaches near 100%, but validation accuracy stagnates and then drops. I suspect overfitting is the issue.

Here’s a simplified version of my training code in PyTorch:

import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms

# Dataset
train_dataset = datasets.ImageFolder('data/train', transform=transforms.ToTensor())
train_loader = torch.utils.data.DataLoader(train_dataset, batch_size=32, shuffle=True)

# Simple CNN
class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding=1)
        self.pool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(16*32*32, 10)

    def forward(self, x):
        x = self.pool(torch.relu(self.conv1(x)))
        x = x.view(-1, 16*32*32)
        x = self.fc1(x)
        return x

model = SimpleCNN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)

# Training loop
for epoch in range(10):
    for inputs, labels in train_loader:
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()

I’ve read about techniques like data augmentation, dropout, and weight regularization, but I’m not sure how to integrate them effectively.

What strategies or best practices would you recommend for reducing overfitting in small datasets while training CNNs?

0 1 105 1 month ago

Subscriber

Maitrik

June 30, 2026

How are you handling memory optimization in large-scale deep learning models?

With newer models getting larger (especially in LLMs and multimodal setups), memory constraints are becoming a major bottleneck during training and inference. Looking for practical approaches others are using to manage this, such as: Gradient checkpointing vs mixed precision Model sharding or distributed training strategies Efficient data loading and batching Would be useful to understand(Read More)

With newer models getting larger (especially in LLMs and multimodal setups), memory constraints are becoming a major bottleneck during training and inference.

Looking for practical approaches others are using to manage this, such as:

Gradient checkpointing vs mixed precision
Model sharding or distributed training strategies
Efficient data loading and batching

Would be useful to understand what’s working in real-world implementations and where trade-offs are being made.

0 2 241 4 months ago

Subscriber

Rob Willoughby

April 22, 2026

Why does my neural network overfit despite using dropout and early stopping?

I’m training a simple deep learning model, but it still overfits even after applying dropout and early stopping. Training accuracy is high, but validation performance drops. import tensorflow as tffrom tensorflow.keras import layers, models model = models.Sequential([layers.Dense(128, activation=‘relu’, input_shape=(20,)),layers.Dropout(0.5),layers.Dense(64, activation=‘relu’),layers.Dense(1, activation=‘sigmoid’)]) model.compile(optimizer=‘adam’,loss=‘binary_crossentropy’,metrics=[‘accuracy’]) history = model.fit(X_train, y_train,validation_data=(X_val, y_val),epochs=50,batch_size=32) What are the common reasons this(Read More)

I’m training a simple deep learning model, but it still overfits even after applying dropout and early stopping. Training accuracy is high, but validation performance drops.

import tensorflow as tf
from tensorflow.keras import layers, models

model = models.Sequential([
layers.Dense(128, activation=‘relu’, input_shape=(20,)),
layers.Dropout(0.5),
layers.Dense(64, activation=‘relu’),
layers.Dense(1, activation=‘sigmoid’)
])