I assume the critical path to be the 32:1 mux for reading from the
register file. Instead of 32:1 mux try using smaller muxes (like 4:1)
in a tree structure and place registers between each stage. This will
increase the read latency.