句子没有任何问题。因为句子中的 yesterday 是修饰不定式完成式的。

句子意思是：据说他昨天读了两本书。

注意以下同义转换：

He is said to have read two books yesterday. 据说他昨天读了两本书。

= It is said that read two books yesterday. 据说他昨天读了两本书。

又如：

Everything seems to have been built yesterday. 好像一切都是昨天才建成的。

He is believed to have crossed the frontier last night. 相信他昨天晚上已经越过了边界。