Is scuba diving dangerous if you have a ventricular septal defect?

Yes. Scuba diving can be extremely dangerous if you have a ventricular septal defect (VSD), even a trivial one.

During ascent from a dive, the expanding gases within the body can pass directly from the left heart chambers to the right heart chambers through the VSD bypassing the pulmonary circulation and the lungs, causing decompression illness (DCI) --colloquially known as the "chokes"-- within minutes. Rapid ascent causes the nitrogen bubbles to form rapidly, blocking blood flow to vital organs, which can result in sudden death as a consequence.

For this reason, individuals with any form of intracardiac right-to-left shunt, including a VSD, are absolutely contraindicated from scuba diving as the risk of catastrophic consequences or even death is so high.