The carbon fibers were pre-treated by means of ultrasonic irradiation in water, immersion in nitric acid without and with ultrasonic irradiation, respectively. Then the effect of the surface pre-treatments on the microstructure, surface chemical and phase composition, the multifilament tensile strength of carbon fibers as well as the microstructure and mechanical property of the carbon fiber-reinforced epoxy resin composites were investigated. Results show that carbon fiber surfaces can be modified effectively by nitric acid oxidation with ultrasonic irradiation. Thereinto, the nitric acid treatment increases the roughness and the amount of oxygen-containing functional groups of carbon fiber surfaces, while the ultrasonic irradiation leads to a good dispersity of carbon fibers and the increase of the specific surface area and the oxygen-containing functional groups of carbon fiber surfaces. Moreover, the combined effect of the nitric acid oxidation and the ultrasonic cavitations could enhance the oxidizing and etching of the carbon fiber surfaces, which consequently may act as "mechanical anchor" and chemically active sites for enhancing the bonding between carbon fiber and resin matrix, and which further improves the mechanical property of the resulting composites significantly.