The problem of access control on outsourced data to "honest but curious" cloud servers has received considerable attention, especially in scenarios involving potentially huge sets of data files, where re-encryption and re-transmission by the data owner may not be acceptable. Considering the user privacy and data security in cloud environment, in this paper, we propose a solution to achieve flexible and fine-grained access control on outsourced data files. In particular, we look at the problem of defining and assigning keys to users based on different attribute sets, and hiding access policies as well as users information to the third-party cloud servers. Our proposed scheme is partially based on our observation that, in practical application scenarios each user can be associated with a set of attributes which are meaningful in the access policy and data file context. The access policy can thus be defined as a logical expression formula over different attribute sets to reflect the scope of data file that the kind of users is allowed to access. As any access policy can be represented as such a logical expression formula, fine-grained access control can be accomplished.