Experiment Runs

Encoder Integration with DenseNet and CBAM in Decoder

Applied a DenseNet block within the encoder and inserted a CBAM module at the decoder's first layer. Training used the CurveVelA dataset (24,000 samples) with a learning rate of 0.001. Loss functions tested were Smooth L1 and LPIPS. Results underperformed compared to baseline InversionNet; LPIPS scores indicated notable degradation, and visual output lost inter-layer contour clarity. Likely cause lies in suboptimal loss formulation. Full-scale training yielded minimal gains and incurred high computational cost, prompting a shift to smaller subsets for efficient iteration.

Network Evaluation on 5,000 Samples Using CurveFaultA

Conducted three comparative setups with an L1 + L2 loss and learning rate 1e-4:

1. DenseNet encoder + CBAM on every decoder layer – Feature reuse via dense links captured rich representations, yet per-layer CBAM addition offered negligible accuracy improvement while increasing compute load.
2. DenseNet encoder + CBAM only at first decoder layer – Achieved highest metrics and visual quality across all tests, surpassing InversionNet on every measure.
3. DenseNet encoder alone – Observed irregular loss behavior, potentially due to dataset variance; loss dropped sharply near epoch 100.

Loss Function Benchmark on 5,000 Samples

Evaluated MAE and Log-Cosh losses on CurveFaultA (lr = 1e-4). This configuration delivered the best performance among tested variants.

Machine Learning Concepts Reviewed

Covered fundamentals including activation functions, softmax operation, and handling multi-output classification tasks.

Mathematical Constructs Practiced

Worked with summation, product notation, definite integrals, min operator, and argmin formulations.

Inspecting Layer Names in a Model

Two practical approaches:

Method 1: Direct model print

class CompactDenseNet(nn.Module):
    def __init__(self):
        super().__init__()
        self.initial_block = ConvBlock(...)
        self.core_block = DenseBlock(...)
        self.compress = TransitionLayer(...)

model = CompactDenseNet()
print(model)

The printed hierarchy reveals nested modules and their parameters.

Method 2: Graph node enumeration

Using a helper like get_graph_node_names:

from utils.graph import get_graph_node_names
nodes = get_graph_node_names(model)
# Example output (input marked as 'x'):
# ['x', 'initial_block.layers.0', 'initial_block.layers.1', ..., 
#  'core_block.layers.5.cat', 'compress.transition.0', ...]

This yields traversal paths useful for targeted hook registration or debugging.

Observed Issues and Resolutions

• Training time instability: Epoch duration rose from ~4m20s to ~4m52s despite smooth loss decline. Potential link to loss design impacting internal operations.

• Image generation crash: Fail to allocate bitmap error traced to memory exhaustion from hidden plt.show() invocations. Fix:

  import matplotlib
  matplotlib.use("Agg")
  

• Post-training PNG display failure: Triggered by prior Agg backend switch. Adjusted show.py to use an interactive backend:

  matplotlib.use("TkAgg")
  

• Loss function design challenge: Identifying robust composite losses remains unresolved.

Upcoming Steps

Initial paper draft deferred beyond July 17 due to pending experiments. Remaining tasks include systematic learning rate sweeps and evaluation on CurveVelA. Large-scale runs will follow on workstation. Emphasis will be placed on rationale behind architectural choices and loss configurations.