The performance of neural network is largely dependent on their capability to extract very discriminant features supporting the characterization of abnormalities in the medical image. Several benchmark architectures have been proposed and the use of transfer learning has further made these architectures return good performances. Study has shown that the use of optimization algorithms for selection of relevant features has improved classifiers. However continuous optimization algorithms have mostly been used though it allows variables to take value within a range of values. The advantage of binary optimization algorithms is that it allows variables to be assigned only two states, and this have been sparsely applied to medical image feature optimization. This study therefore proposes hybrid binary optimization algorithms to efficiently identify optimal features subset in medical image feature sets. The binary dwarf mongoose optimizer (BDMO) and the particle swarm optimizer (PSO) were hybridized with the binary Ebola optimization search algorithm (BEOSA) on new nested transfer functions. Medical images passed through convolutional neural networks (CNN) returns extracted features into a continuous space which are piped through these new hybrid binary optimizers. Features in continuous space a mapped into binary space for optimization, and then mapped back into the continuous space for classification. Experimentation was conducted on medical image samples using the Curated Breast Imaging Subset of Digital Database for Screening Mammography (DDSM+CBIS). Results obtained from the evaluation of the hybrid binary optimization methods showed that they yielded outstanding classification accuracy, fitness, and cost function values of 0.965, 0.021 and 0.943. To investigate the statistical significance of the hybrid binary methods, the analysis of variance (ANOVA) test was conducted based on the two-factor analysis on the classification accuracy, fitness, and cost metrics. Furthermore, results returned from application of the binary hybrid methods medical image analysis showed classification accuracy of 0.8286, precision of 0.97, recall of 0.83, and F1-score of 0.99, AUC of 0.8291. Findings from the study showed that contrary to the popular approach of using continuous metaheuristic algorithms for feature selection problem, the binary metaheuristic algorithms are well suitable for handling the challenge. Complete source code can be accessed from: https://github.com/NathanielOy/hybridBinaryAlgorithm4FeatureSelection