There would be a significant danger involved in diving with displaced, fractured ribs (and I don't think there are many people stoic enough to try it). One of the complications of rib fracture is puncture of the lung (pneumothorax), and a pneumothorax at depth is a potentially lethal injury. There is some not completely theoretical danger of a cracked rib becoming displaced with activity or impact and doing the same thing, which is why I think most physicians would say no diving with a nondisplaced rib fracture, either (i.e. cracked rib).
A costal cartilage injury does not have this potential complication. There, you're just dealing with pain and the anxiety related to it, and to perhaps feeling as though you can't get a deep breath.
Archman's point about the instructor being complicit in your decision to dive with an injury is a very good one.