Hi Devin, nice write up as usual. Re footnote 9 it looks like the convolution might not really be typical convolution (given it's being applied to specific elements alone?) and it's more the pooling that effectively makes the behaviour "set-like" through aggregation as opposed to an ordered calculation.
To add to the possible player order invariance options there's an interesting paper deploying set-attention which looks like a promising direction you may be interested in:
Hi Oscar. Thank you, really interesting! Honestly, I think I'd need some visuals to fully grasp but the pooling makes sense since it's along the player axis.
Hi Devin, I'm actually new to this as a concept too so have had a look this morning and these materials seemed to visually explain the concept best for me:
Video here with a good image and some good explanation. Looks like at 5:50 it's basically the architecture being talked about in the BDB architecture:
Appears to be a method that can be useful for dimensionality reduction and in this case they've reduced their input matrix by using pointwise convolution along the defensive axis to produce 3 different values per player which I think are then pooled with the mix of mean and max they mentioned. I'm not 100% sure (think they probably could/should have made their viz easier to follow) but I think that's what they're doing! (I'm actually a bit confused as to why you would use max rather than min here - surely the most relevant information for blockers would be who's closest to your position & velocity rather than most dissimilar? Maybe I'm misunderstanding.)
Hi Devin, nice write up as usual. Re footnote 9 it looks like the convolution might not really be typical convolution (given it's being applied to specific elements alone?) and it's more the pooling that effectively makes the behaviour "set-like" through aggregation as opposed to an ordered calculation.
https://www.kaggle.com/c/nfl-big-data-bowl-2020/discussion/119400#727631
To add to the possible player order invariance options there's an interesting paper deploying set-attention which looks like a promising direction you may be interested in:
https://arxiv.org/abs/2410.17785
Hi Oscar. Thank you, really interesting! Honestly, I think I'd need some visuals to fully grasp but the pooling makes sense since it's along the player axis.
Hi Devin, I'm actually new to this as a concept too so have had a look this morning and these materials seemed to visually explain the concept best for me:
Video here with a good image and some good explanation. Looks like at 5:50 it's basically the architecture being talked about in the BDB architecture:
https://youtu.be/wf2HblQbP-U?si=ur4hC3d8aO5g8BBg&t=265
Brief summary and image of an individual 1x1 convolution (with channel/feature depth of 32 in this case):
https://paperswithcode.com/method/1x1-convolution
Video on groups that includes (1,1) convolution aka "pointwise convolution" with nice viz:
https://youtu.be/vVaRhZXovbw?si=s3jlj6800K4Cyy9R&t=182
Gif from same person, second half has pointwise convolution:
https://animatedai.github.io/media/depthwise-separable-convolution-animation-3x3-kernel.gif
Appears to be a method that can be useful for dimensionality reduction and in this case they've reduced their input matrix by using pointwise convolution along the defensive axis to produce 3 different values per player which I think are then pooled with the mix of mean and max they mentioned. I'm not 100% sure (think they probably could/should have made their viz easier to follow) but I think that's what they're doing! (I'm actually a bit confused as to why you would use max rather than min here - surely the most relevant information for blockers would be who's closest to your position & velocity rather than most dissimilar? Maybe I'm misunderstanding.)
Hope the pictures help at least!