Looking at UW Madison
Arabic TAs. Visit office hours for Arabic TAs (see Language at UW: Arabic or African Cultural Studies: Arabic. Office hours may simply provide an opportunity to practice speaking MSA and asking questions about grammar (which would not normally be done with a mentor). Alternatively, you may find that a TA is willing to meet with you regularly outside of office hours and serve as your mentor. They also may know of other qualified individuals who would be willing to do so.
Fulbright Fellows. Scour department websites and announcements to see if there are Fulbright fellows from Arabic-speaking countries. These individuals may be open to mentoring you, or they may not. Source the African Studies Program first as these individuals are may be more enthusiastic at the prospect of being a language mentor. Also, check out staff listed at the African Cultural Studies Department and the Arabic-language program staff and faculty here.
African Studies Program. Talk to staff in the Africa Studies Program (e.g. the instructor of Africa 671!) and ask if they know of anyone in the department (or elsewhere) that might be qualified for and interested in serving as a language mentor.
Arabic Roundtable. A group of students enrolled in 3rd and 5th year Arabic meets with an Arabic TA each evening Monday through Thursday from 5-6pm. I have found the group to be welcoming, focused, and diligent in only speaking in Arabic (for the most part). Through this group, you may find potential mentors (e.g. the TA, friends of the TA, friends of the students). Even if you do not find a mentor, the time spent speaking Arabic will certainly be fruitful. You can view the calendar here.
Looking into the Madison community
Be mindful of Arabic varieties/dialects when sources the community for potential mentors. Be sure that you are connecting with individuals that are fluent in MSA or the variety in which you are interested.
Madison Restaurants. Speak to the owners of restaurants that serve food endemic to Arabic-speaking countries. For example, I have had luck speaking to the owner of Buraka, an Ethiopian restaurant on Willy St.
MATC Arabic Courses. Contact the instructor for the MATC Arabic courses. As with the Arabic TAs (above), this individual may be a potential mentor, or may know of qualified and interested individuals.
African and American Store. Stop in this shop on the corner of East Wash and East Johnson. The proprietor is a lovely woman who will be more than happy to help you. I have not found a mentor through this connection yet, but I know that Mariam is well connected and she may connect me with a potential mentor in the future. As with Madison Restaurants (above), any person I connect with through the African and American Store may only speak a variety/dialect of Arabic (which can still be very helpful!).
Final Note: In all your searching, consider whether it will be best to work with a male or female mentor. You may decide that it does not matter enough to affect your choice of language mentor, but at least give it some thought.