Restructuring source code structure, such as moving and renaming classes or methods, are inevitable activities in software develop- ment, and are recommended for the improvements of maintainabil- ity. However, it has been not easy to understand properly what log- ical structural changes occur. This is because we can obtain only file-level and line-level information from source code management systems about changes. This paper presents a technique of such in- ferring restructuring operations on logical structure of Java source code. For inferring structural change operations, the core part is mapping elements between two revisions. Previous related studies tackle this problem based on the analysis of subgraph similarity, which takes lots of time. We find match candidates based on the similarity of element contents and identify matches with Bayesian inference based on empirical data. We report the result of empiri- cal evaluation of our technique with open source software projects from Android and Eclipse. We see that our technique identify most element matches correctly and provide appropriate operations, and it took only a few seconds to analyze entire history of each project.