And by "handled within the kernel", you mean like a big switch to execute the right thing depending on format ?
I guess you know at invocation time the image fmt so you could adapt the program at that time. I'd assume a lot of kernel are always run with the same types as input. But not sure if that's something that fits in the current "infrastructure".