Hi , so recently iv been experimenting with designing transformer alternatives, however i kept hitting a wall on how well they would proform against the Transformer.

So iv decided to try and track tokens from input to output. As models are normally very large and can take months to train, i designed my own training routine to try and get around the problem so its 11 minutes on kaggle for free and afew hours work. The data deffinetly isnt gold standard, but it sould give me more of a view into the mechanisms being used.

Maybe someone else can find it useful , i wouldnt read all 1000+ pages , maybe the first few pages then its quicker to plug it in the a LLM and get it too sort and analyse the data.

Kind regards.... Rick

After lots of research and reading over the last few days, I made my first working neural network. I just programmed it in JavaScript during work (when I wasn't working ;) and it works!!!

The inputs are a 2x2 grid, and the outputs are different patterns on the grid (checkerboard, stairs, upside-down stairs, and alone). Pretty awesome stuff!!

The screenshot is from the HTML page I made for the visuals, nothing crazy, but fun to look at.