The answer to this question strongly depends on your area of expertise. As a rule of thumb, professions in which you could potentially harm people (doctors, teachers, engineers, lawyers, etc.) need to seek accreditation of their diplomas. Usually these jobs are within
a list of regulated professions. If you are an Economist or a specialist in IT, you may find a job without the official accreditation. You will still need to have documents translated, though. You will find
a complete guide to diploma accreditation on the official site of the Ministry of Education and Research (BMBF).