pay a visit to = visit (前一个visit 是名词,a visit 作pay的宾语,to 是介词;后一个 visit 是及物动词)。所以,正确的句子应该是:
That's the factory _to which__ we paid a visit last year.(定语从句中不缺乏地点状语,所以不能选择A. where)
如果是这个题目,就应该选择A. where:
That's the factory _____ we worked last year.
A. where B. to which C. which D. in which