March 13, 2024 at 03:32PM
In late 2023, a comparison of ChatGPT and Google Bard’s performance in handling writing security policies is discussed. Both tools are evaluated across various use cases for information security professionals, such as generating diagrams, explaining architecture, interpreting exploit code, and writing policies. Each tool showcases strengths and weaknesses in different areas.
Based on the meeting notes, the comparison between ChatGPT and Google Bard (now Gemini) for handling writing security policies and other information security use cases reveals the following:
1. Generating Diagrams or Concept Flows: ChatGPT produces nonsensical and illegible outputs for diagram generation, whereas Gemini provides usable ASCII diagrams.
2. Explaining Architecture Diagrams: Google Gemini is more succinct and better at explaining architecture diagrams compared to ChatGPT.
3. Interpreting Exploit Code: Both tools identify and explain exploit code effectively.
4. Interpreting Log Files: Gemini explains log files better and provides clear summaries and follow-up steps compared to the verbose output of ChatGPT.
5. Writing Policies and Security Documentation: Both tools perform consistently, but Gemini generates better security documentation than ChatGPT.
6. Identifying Vulnerable Code: ChatGPT correctly identifies vulnerabilities with a prompt, while Gemini identifies some vulnerabilities and proposes amended code to fix them.
7. Writing Scripts and Code: Both tools produce clear and readable code for the given prompt.
8. Analyzing Data and Metrics: ChatGPT offers an advantage through its Data Analyst plugin for data analysis and visualization, while Gemini is limited to guiding users through Excel and Power BI.
9. Writing User Awareness Messages: Gemini wins in generating security awareness emails with the right tone and brevity, while ChatGPT also performs well but tends to be slightly longer.
10. Interpreting Compliance Frameworks: Gemini outperforms ChatGPT by correctly listing specific requirements from the standards for interpreting significant changes under PCI-DSS.
To summarize, depending on the specific use case, either ChatGPT or Google Gemini can be valuable in boosting productivity and assisting with day-to-day activities in the field of information security.