How to kill wasps that've built a nest INSIDE my sealed solar garden lights?

You will need to search well and find the point of entry. The wasps typically don't make a lot of effort to find their home. There is a hole or vent in the light the wasp is going in. If you can find it and spraying in it doesn't work, try tying a plastic bag around it. This should suffocate them. Just have can of wasp spray available in case their friends come to visit them when you are trying to kill them.