No penalty for programming J#.
Contrary to rumors that I've heard, I can find no runtime penalty for code written in J# as opposed to other .NET languages and specifically C#. The one significant difference I've seen in IL generated by the vjc (J#) and csc (C#) command-line compilers is that all Java instance methods are (correctly) virtual, while C# methods default (correctly) to non-virtual. That's a difference in the languages' designs that might lead to a measurable performance difference in the speed with which methods are called (perhaps that is what is going on in the Cholesky benchmarks?).
Something I'd heard, which turns out to be incorrect, is that J#'s implementation of inner classes was flawed. Inner classes are real nested classes at the IL level. So, if you prefer anonymous inner classes to delegates, J#-away with no hesitation.