We demonstrate a novel in vivo sampling device for preparation-free quantitative detection of residual malachite green in fish with depth profiles. The device is a surface-enhanced Raman scattering (SERS)-active acupuncture silver needle carved with an array of micrometer-sized grooves for trapping polyvinylpyrrolidone (PVP)-stabilized Au nanoparticles (Au NPs). The microgrooves not only prevented the absorbed Au NPs being erased during insertion but also provided observation/locating convenience under the optical microscope of the Raman spectrometer. Moreover, the Au NPs anchored onto the Ag needle help improve the sensitivity of the SERS measurements, while the plasmon coupling between the Ag needle and Au NPs greatly amplifies the SERS signal. Furthermore, the PVP on the surface of the Au NPs serves as an inherent internal calibration standard leads to reduced SERS signal fluctuation, holding promising for quantitative SERS detection. The fabricated Au NPs-groove-trapped Ag needles (Au-G-AgNs) are further applied for in vivo detection of malachite green, a prohibited carcinogen commonly found in aquaculture products that are expensive and complicated to detect by current techniques. The result shows that in vivo detection of malachite green at an ultralow concentration of 0.1 nM is achieved, demonstrating the capability of the SERS sampler for ultrasensitive in vivo detection with minimal invasion.