Acceleration is an important feature used to characterize the vibration of mechanical equipment. In order to reduce power frequency interference, a non-metallic base is usually added to the vibration acceleration test system between the sensor and the measuring point. Three types of non-metallic sensor bases commonly used in the field of ship vibration testing were analyzed in this paper. The transfer function of the sensor and base system was derived by the impedance method, and an analysis was performed to determine how the base stiffness influenced the effective frequency band of the sensor. Through numerical calculation and experimental comparison of three types of non-metallic sensor bases, the relationship between effective testing frequency band and impedance matching was built. The results show that if a non-metallic base is added to the sensor, the test structure will produce additional modes. If the base is less rigid, the frequencies of the additional modes will drop into the effective test frequency band of the sensor, which can seriously impact test accuracy. Selecting a non-metallic base with larger stiffness helps to ensure the validity of the test results.