I do think there needs to be a level requirement, but lvl 35 is higher than it should be. Maybe lvl 25 or lvl 30 would be a more appropriate permanent range for raids.
I'm sure by starting at lvl 35 they are testing the system using players who have show enough commitment to get there and are unlikely to quit if there are problems with the initial launch. Still a high enough player pool to get the kinks worked out but small enough and dedicated enough that they won't be overly discouraged if there are issues, plus that group of players will have strong enough Pokémon to see how things shake out against the beasts in the raids.
Adjustments will be made to the gym and raid system, so I hope people give it time to develop.