More or less, yeah. And more things than just Haste/Slow affect your bar.
Sounds neat, like in FFIV (Though, in that game you needed perfect timing to execute some commands), also, i have been reading some battle systems and this one seems interesting
Conditional Turn-Based
The Conditional Turn-Based Battle system, or the Count Time Battlesystem in Japan, designed by Toshiro Tsuchida is used in Final Fantasy X. CTB is a turn-based system which does not operate in rounds, instead it uses an Act List that is affected through various means and thus does not guarantee that each participant in a battle will have an equal number of turns. Units with higher speed take more turns than slower ones, making speed more important than in other turn-based battle systems. Players can substitute party members mid-battle adding a new level of strategy.
Spells and abilities (such as Haste or Overdrives) modify the Act List, as some abilities require a longer cool down time. Weaker abilities tend to require less cool down time, thus introducing a trade-off between speed and power. When a character's turn begins all action stops while the player decides upon an action. This shifts the focus from reflexes and quick decision-making to strategy and careful planning
I think that a method to change your party during a fight seems logical, Chrono Trigger only had the possibility to bring 3 fighters at once, and you could only change them in the overworld, FFVI had the feature of multiple characters in different parts of the world, but at the end of it, everyone groups together and you have the possibility of choosing anyone you want for the final battle.
The neat thing about the FFX system is that you could wait quite some time until you execute one turn, so, that command itself adds a little more of variety and strategy, while in other games the enemy could defeat you even before pressing a button, Chrono Trigger suffered from that, this could add stress by the fact that you have to execute a command before the enemy do something.
So, in a certain kind of way, you can't predict what kind of moves the enemy will launch, so, giving player an unlimited amount of time until they select a command (Let's say, when you press a command you attack immediately, but the enemy won't damage you unless he has some kind of counter to physical/spells attacks) and when you have all that you needed in the same turn, the enemy attacks.
But in my perspective, enemy speed does matter, i think that by having one turn per one team it's enough for everyone, if the enemy team engages you first, or they have an skill to always attack first, this could work better as you need to think before you act something, if let's say, enemy speed/agility is top notch, but it has a mediocre defense/special you could throw the enemy a spell that reduces the attack of the enemy, and therefore, you will only need to attack a weak attack that has more precision than powerful attacks (Like FE Weapon System)