This paper presents a contrastive deep learning-based wheelability assessment method bridging street-scale smartphone point clouds and a city-scale 3D pedestrian network (3DPN). 3DPNs have been studied and mapped for walkability and smart city applications. However, the city-level scale of 3DPN in the literature was incomplete for assessing wheelchair accessibility (i.e., wheelability) due to omitted pedestrian paths, undetected stairs, and oversimplified elevated walkways; these features could be better represented if the mapping scale was at a micro-level designed for wheelchair users. In this paper, we reinforced the city-scale 3DPN using smartphone point clouds, a promising data source for supplementing fine-grain details and temporal changes due to the centimeter-level accuracy, vivid color, high density, and crowd sourcing nature. The three-step method reconstructs pedestrian paths, stairs, and slope details and enriches the city-scale 3DPN for wheelability assessment. The experimental results on pedestrian paths demonstrated accurate 3DPN centerline position (mIoU = 88.81 %), stairs detection (mIoU = 86.39 %), and wheelability assessment (MAE = 0.09). This paper contributes an automatic, accurate, and crowd sourcing wheelability assessment method that bridges ubiquitous smartphones and 3DPN for barrier-free travels in high-density and hilly urban areas.