Accuracy and efficiency are very much required in medical diagnosis though it is an important at the same time complicated task. The automated medical diagnosis system would be highly beneficial.This automation removes unwanted biases, errors and costs which affects the quality of clinical diagnosis. Data mining techniques especially Decision Trees play an efficient role in the classification of medical data. To know the best decision tree classifier for medical data sets various frequently used decision tree algorithms are compared based on their classification accuracy. As per the statistics of National Cancer Institute Breast cancer is a leading cause of death among females in economically developing countries and a second cause in developed countries.Early detection of breast cancer will reduce the death rate.In order to extract the most relevant features from the data sets various Feature Selection methods and Hybrid Approaches with Ensemble techniques ie., Bagging and Boosting with decision tree classifier on breast cancer data sets are studied. And a new hybrid algorithm is proposed with cascading Feature selection, Clustering and Classification to enhance the accuracy.