Get the parameter size of AnalogContext correctly

[Zhaoxian-Wu]

It seems impossible to access the weight size in the parameters. To be specific, I run the following code:

from aihwkit.nn import AnalogLinear
model = AnalogLinear(784, 256)
for param in model.parameters():
    print(type(param), param.size())

it outputs that

<class 'torch.nn.parameter.Parameter'> torch.Size([256])
<class 'aihwkit.optim.context.AnalogContext'> torch.Size([])
<class 'aihwkit.optim.context.AnalogContext'> torch.Size([])

The param size is incorrect here ([784, 256] is wanted but [] is returned).

I understand that to access the correct weight, we should use the following snippet instead

weight, bias = model.get_weights()
print(weight.size(), bias.size())

and we get

torch.Size([256, 784]) torch.Size([256])

But do we have a more convenient way to the size of the linear layer through param?

[maljoras]

Hi @Zhaoxian-Wu,
The analog crossbar arrays are accessible by the generatores "analog_tiles" and "named_analog_tiles" instead, as they are not parameters in the torch sense (they are handled separately from the torch auto-grad to be able to implement a custom update).

for name, tile in model.named_analog_tiles():
     print(name, tile.get_x_size(), tile.get_d_size())

This will get the individual crossbar sizes (there might be more than one used for a layer).

To get the size of the layer you can simply use: print(model.in_features, model.out_features) for the linear layer.

Another alternative is to get the full weights of the layer (however, that might involve a copy and should be avoid for performance reasons): model.get_weights().shape

[Zhaoxian-Wu]

Thanks for your patient reply. It seems it should be an elegant way to access the size. However, when I tried this snippet the package raised an error saying there is no get_x_size()

model = AnalogLinear(784, 256)
for name, tile in model.named_analog_tiles():
     print(name, tile.get_x_size(), tile.get_d_size())
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[176], line 4
      2 model = AnalogLinear(784, 256)
      3 for name, tile in model.named_analog_tiles():
----> 4      print(name, tile.get_x_size(), tile.get_d_size())

File [~/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py:1614](https://file+.vscode-resource.vscode-cdn.net/Users/wangbudong/Library/CloudStorage/OneDrive-Personal/2023.09.05%20analog/~/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py:1614), in Module.__getattr__(self, name)
   [1612](file:///Users/wangbudong/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py?line=1611)     if name in modules:
   [1613](file:///Users/wangbudong/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py?line=1612)         return modules[name]
-> [1614](file:///Users/wangbudong/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py?line=1613) raise AttributeError("'{}' object has no attribute '{}'".format(
   [1615](file:///Users/wangbudong/anaconda3/envs/analog/lib/python3.11/site-packages/torch/nn/modules/module.py?line=1614)     type(self).__name__, name))

AttributeError: 'AnalogTile' object has no attribute 'get_x_size'

It seems that the function get_x_size is defined only for class SimulatorTile and TorchSimulatorTile but tile is neither of them

from aihwkit.simulator.tiles.base import SimulatorTile
from aihwkit.simulator.tiles.torch_tile import TorchSimulatorTile
print(isinstance(tile, SimulatorTile))
print(isinstance(tile, TorchSimulatorTile))
---------------------------------------------------------------------------
False
False

[maljoras]

Note that there is also a summary tool:

from aihwkit.utils.analog_info import analog_summary
analog_summary(model)

This will print out the later sizes and crossbar configurations etc.