You have to deduct TDS as per section 51 of the CGST Act , if he is supplying taxable goods or services irrespective of the registration. please refer section 51 and confirm.
Nothing has been specified in the section 51, regarding to deduct TDS only for registered supplier only the "supplier of taxable goods or services" word's have been used. You Can Contact to CBIC help disk number.