Awesome work with Excel there, that makes things really easy to see. I used to be able to do that but it's been been 5+ years since I tried anything more advanced than "Auto-SUM"
My contention is that you cannot compare HP-to-HP, as you have done and as DL-North said on page 1... that is comparing Apples-to-Armadillos. Cheaper gas engines make more HP than more expensive diesel engines.
2018 RAM 1500 5.7L Gas 395 HP -$3,120
2018 RAM 1500 3.0L Diesel 240 HP
2018 RAM 2500 6.4L Gas 410 HP -$8,700
2018 RAM 2500 6.7L Diesel 370 HP
2018 Chev Colorado 2.5L Gas 200 HP -$6,290
2018 Chev Colorado 3.6L Gas 308 HP -$3,105
2018 Chev Colorado 2.8L Diesel 181 HP
What does your Excel graph look like if you compare the liquid-cooled Kohlers?
LH775 0.75L Gas 30 HP -$746
KDW1003 1L Diesel 23 HP
As it relates to automobiles, current diesel emissions regulations are more stringent than gasoline in regard to particulate emissions, requiring expensive emissions equipment. Diesels are also viewed, in trucks at least, as "better"...people will pay more for them, and I suspect the pricing structure is used by the automakers to improve the profit margins on those vehicles.
I pulled the curves for those 2 liquid cooled engines (full disclosure: I had to use the curve for the LDW1003, which is the Lombardini-branded version of the KDW, because I couldn't find a proper KDW curve. I used the middle curve again.) and overlaid them with the previous 2. Result is below. Torque and horsepower on the 30 hp gasser don't pass the 23 hp diesel until 2700 rpm, which is more engine speed than you'd likely use for any non-PTO task.
When it comes to engine power, the transmission and final drives for the tractor need to be designed to handle max power. More power requires larger tractor components to maintain reliability. This is why you don't see a 30 hp in the smallest size of SCUT, be it the Deere 1 series, MF 1700, Kubota BX80, or New Holland Boomer Compact. You can buy heavier tractors with low power engines, but those tractors cost more. In the smallest SCUT class, a 30 hp gas engine wouldn't be available.
Given that tractors have components that are sized by max power, are sold by horsepower, and that 1 horsepower does the same amount of work regardless of what energy source it came from, comparing engines of equal horsepower is what the manufacturer does when looking to design a machine. As the curves show, diesels generate better power through the low end of the RPM range, making them more suited for the way the machine is actually used. The manufacturer (any company, really) is worried about 1 thing, and that's making money in a sustained fashion. One way to do that in the durable goods market is to make things that last past the warranty period far enough to keep people happy, minimize complaints that cause people to bring equipment in for "warranty" fixes, and maximize word-of-mouth (free) advertising. An engine that lasts longer, provides a better operating experience at speeds below full throttle, and does better on fuel is an easy choice for the OEM in light of their goals.
In the world where tractor enthusiasts like us do things like repower old GTs, the concept gets muddy, as many old machines were well over-built and can handle some additional power. You or I might well do that on a repower and get away with it because we understand how to take care of our equipment. In the repower circumstance, where the cost of the tractor is small, and resale value is poor, the cost of the engine becomes a lot more important to the decision. In that case, gas is an excellent choice. For what you're trying to accomplish, gas may also be an excellent choice. Unfortunately, for the original equipment manufacturer in today's market (being the ones who determine what is available on the market), I don't see any case where selling a gas SCUT would help them make any more money, which is the only thing that they look at. When it doesn't hit the bottom line, they won't build it.