I have traveled back to April 23 and went forward from there until i encountered Leif 3 times in which i bought a lot from him (10 of everything each time). 4th time I meet him he still does not give me the DIY for the fence. Also the 23  of April is not announced as nature day or anything.

Has something happened to time traveling og am I doing something wrong??

In New Horizons holidays don't work when you time travel. They have to have a time zone to verify the date so it works.