This topic is always an important topic, which always split the different it departments. If we look the CIO's importance list, the cost is often more important than the sophistication of the architecture. The root cause analysis often brings us to HR level questions? If we have multiple departments, like network administration, and SAN administration, they always want to protect their land. If we are more realistic, we can identify that we can simplify the number of different networks.
In my view the main objective which we can save the most is the SAN network. If we look into the past years of the developments, we can see the development of SAN loose against the current other technologies.
Anyone who knows me, know that I'm searching solutions which currently used in the HPC sector. From this view, it is a clear fact that, for interconnect technology I prefer Infiniband.
Why? Because of low latency, and high throughput. And if we look for the price tag we can identify easily that, these prices comparable to the 10Gb interconnects.
Mellanox technologies currently have 100Gb connection speed in their ConnectX-4 infiniband HCAs, and in their SB7700 switches. If we look for the 200ns switch port-port latency we can easily realize that these latencies comparable to the local drives. From this perspective, we can use any number of disk drives, in a server, with minimal latency costs.
If we look further, we can put more application logic to the storage, like Oracle did in their Exadata Storage cells. With this effort we not only save compute power, but also save energy.
A few years ago Meg Whitman took a presentation, where she has shown a number that 80% of energy goes to interface communication. For me it is a personal interest how we can decrease these losses with more efficient system design.
I believe in that in the future, computing will use much more specialized devices, instead of using traditional servers. If we think about traditional computational tasks, it's not changed in the past (e.g. Calculation of interest rate)
In a traditional processor we need to use multiple CPU cycle to do an operation, if we create specialized processor, we can do an operation with one cycle instead of multiple hundreds / thousands. If we can use these functions with more efficiency, we can decrease clock rate, power consumption. For these kind of developments we can learn a lot from the Bitcoin hardware business. We can compare there the performance/power consumption of different implementations:
A CPU based implementation
A GPU based implementation
An FPGA based implementation
And at last the ASIC implementation
From this perspective, we can identify the space for these kind of developments. If a company has a code / function based monitoring of their systems, they able to identify the TCO decrease, if they invest to their most used code patterns.
With the power cap decrease we will able to put more functionality to one server, and with this effort we can cut down unnecessarily needed communication.
If we look the complete picture, with these efforts, we able to use one converged communication channel. The question is that, what is the bandwidth and latency need.