As a pyraminx solver that currently averages 4.x seconds, I feel I am somewhat qualified to explain how to get where I am, and with enough practice you can get below 3 seconds using my tips.
1: Learn a method type that you want to stick with.
Method neutrality isn't a necessity to get really fast at Pyraminx (sub-3), so you don't need to use up all your time like this. Pick a good method from that type that you know you can use, and practice as much as you can with that method.
2: Learn to one-look.
This is a necessity to be sub-3 at pyraminx. You have to know which moves your going to do during the solve and how to execute them in the most ergonomically friendly way possible. It's quite a difficult task, but once you have this step down there's not much more do do.
3: Learn proper fingertricks.
I'm just saying this because I see a lot of people using awful fingertricks for solving pyraminx. Some even have sub 5 averages, which is funny considering that they could probably drop around 1/3 of a second by just learning good fingertricks. Anyway, learn how to do flicks with your fingers and try to minimize regrips as much as you can.
4: Do tips the right way.
Don't solve all your tips at the beginning or end, and don't solve one at a time with a flip in between. Solve tips while doing the solve, and if you can't solve a few, then solve them simultaneously at the start or end of a solve using your indexes or pinkies (depending on the situation).
This probably does seem like a lot, but these are things I took into account when trying to get good, so combine these with practice and I think that you could be able to reach your goal.