Hey, your problem is likely your MOSFET.
The IRFZ44N has a 2-4V gate threshold. Sometimes overkill will actually make things not work.
This means that somewhere in that voltage range it will start to even turn on. You seem to have at least gotten one with around a 3V Vth.
You are likely driving it in the cutoff region to the very low ohmic/triode. That means that you are controlling how much current is going through proportional to the voltage, but extremely low
If you look at figure 1 on the datasheet, you can see that the 4.5V is pretty much the recommended voltage on the gate (assuming low side switch). A good way to think of it is a ball valve, you are just turning the handle a tiny bit right now when it should be open all the way.
I would try a FET with a lower Vth. Also put a current limiting resistor in series with the IR lED if it isn't there already since they only have a 1.2V Vfwd many times. Don't want to blow it up.