Should I worry about my Lung Nodule? I might be at a higher risk for cancer...?

Well I am sorry that doctor was such an ***! Yes, you do have numerous risks and should be concerned. That being said, you need to have a biopsy of this to make certain it is not cancer. That is the only way to really find out what it is. Take your CT scan and results to a pulmonologist ASAP and get a work up. Don't let them take the wait and see approach.