This is a term borrowed from circuit design, and simply refers to the shortest path from one end of a circuit (or neural network, in this case) to another. It's the critical path, because it's the shortest possible delay, in other words it's the most critical to improve. Improving this path will improve every possible scenario, since this is the most common and core path that everything takes. Again, normally this term is used in circuit design and stuff, but it's applicable to neural networks as well.