Pulling log messages from a MySQL table

The name of the pictureThe name of the pictureThe name of the pictureClash Royale CLAN TAG#URR8PPP





.everyoneloves__top-leaderboard:empty,.everyoneloves__mid-leaderboard:empty margin-bottom:0;







up vote
3
down vote

favorite












I want to get this code reviewed for a function I wrote that pulls log messages from a MySQL table:



def get_logs_messages(request_id, destination=None):
"""
get logs from deployment table
:param int request_id: request id of selected deployment.
:param str destination: destination site
:return: list of messagesfor selected deployment based on request id.
:rtype: list
"""
conn = MySQLdb.connect(**dbconfig)
query = "SELECT deployment_id "
"from deployment "
"WHERE request_id=request_id "
" dest".format(
request_id=request_id,
dest="AND DESTINATION = 'loc'".format(loc=destination) if destination else "")
cursor = conn.cursor()
cursor.execute(query)
data = cursor.fetchall()
cursor.close()
msgs =
for item in data:
query = "SELECT modified_dt , message from scripts_deployment_logs WHERE deployment_id = ".format(item[0])
cursor = conn.cursor()
cursor.execute(query)
result = cursor.fetchall()
for dt, msg in result:
msgs.append("dt: msg".format(dt=str(dt), msg=msg))
cursor.close()
conn.close()
return msgs


I get a request ID from a web interface which is using a REST API to call this function, and this function retrieves the deployment id based on which it displays the logs. I am not very fluent with the MySQL part so I have used mysql.connector with simple SQL queries.







share|improve this question



























    up vote
    3
    down vote

    favorite












    I want to get this code reviewed for a function I wrote that pulls log messages from a MySQL table:



    def get_logs_messages(request_id, destination=None):
    """
    get logs from deployment table
    :param int request_id: request id of selected deployment.
    :param str destination: destination site
    :return: list of messagesfor selected deployment based on request id.
    :rtype: list
    """
    conn = MySQLdb.connect(**dbconfig)
    query = "SELECT deployment_id "
    "from deployment "
    "WHERE request_id=request_id "
    " dest".format(
    request_id=request_id,
    dest="AND DESTINATION = 'loc'".format(loc=destination) if destination else "")
    cursor = conn.cursor()
    cursor.execute(query)
    data = cursor.fetchall()
    cursor.close()
    msgs =
    for item in data:
    query = "SELECT modified_dt , message from scripts_deployment_logs WHERE deployment_id = ".format(item[0])
    cursor = conn.cursor()
    cursor.execute(query)
    result = cursor.fetchall()
    for dt, msg in result:
    msgs.append("dt: msg".format(dt=str(dt), msg=msg))
    cursor.close()
    conn.close()
    return msgs


    I get a request ID from a web interface which is using a REST API to call this function, and this function retrieves the deployment id based on which it displays the logs. I am not very fluent with the MySQL part so I have used mysql.connector with simple SQL queries.







    share|improve this question























      up vote
      3
      down vote

      favorite









      up vote
      3
      down vote

      favorite











      I want to get this code reviewed for a function I wrote that pulls log messages from a MySQL table:



      def get_logs_messages(request_id, destination=None):
      """
      get logs from deployment table
      :param int request_id: request id of selected deployment.
      :param str destination: destination site
      :return: list of messagesfor selected deployment based on request id.
      :rtype: list
      """
      conn = MySQLdb.connect(**dbconfig)
      query = "SELECT deployment_id "
      "from deployment "
      "WHERE request_id=request_id "
      " dest".format(
      request_id=request_id,
      dest="AND DESTINATION = 'loc'".format(loc=destination) if destination else "")
      cursor = conn.cursor()
      cursor.execute(query)
      data = cursor.fetchall()
      cursor.close()
      msgs =
      for item in data:
      query = "SELECT modified_dt , message from scripts_deployment_logs WHERE deployment_id = ".format(item[0])
      cursor = conn.cursor()
      cursor.execute(query)
      result = cursor.fetchall()
      for dt, msg in result:
      msgs.append("dt: msg".format(dt=str(dt), msg=msg))
      cursor.close()
      conn.close()
      return msgs


      I get a request ID from a web interface which is using a REST API to call this function, and this function retrieves the deployment id based on which it displays the logs. I am not very fluent with the MySQL part so I have used mysql.connector with simple SQL queries.







      share|improve this question













      I want to get this code reviewed for a function I wrote that pulls log messages from a MySQL table:



      def get_logs_messages(request_id, destination=None):
      """
      get logs from deployment table
      :param int request_id: request id of selected deployment.
      :param str destination: destination site
      :return: list of messagesfor selected deployment based on request id.
      :rtype: list
      """
      conn = MySQLdb.connect(**dbconfig)
      query = "SELECT deployment_id "
      "from deployment "
      "WHERE request_id=request_id "
      " dest".format(
      request_id=request_id,
      dest="AND DESTINATION = 'loc'".format(loc=destination) if destination else "")
      cursor = conn.cursor()
      cursor.execute(query)
      data = cursor.fetchall()
      cursor.close()
      msgs =
      for item in data:
      query = "SELECT modified_dt , message from scripts_deployment_logs WHERE deployment_id = ".format(item[0])
      cursor = conn.cursor()
      cursor.execute(query)
      result = cursor.fetchall()
      for dt, msg in result:
      msgs.append("dt: msg".format(dt=str(dt), msg=msg))
      cursor.close()
      conn.close()
      return msgs


      I get a request ID from a web interface which is using a REST API to call this function, and this function retrieves the deployment id based on which it displays the logs. I am not very fluent with the MySQL part so I have used mysql.connector with simple SQL queries.









      share|improve this question












      share|improve this question




      share|improve this question








      edited Jun 28 at 2:05









      Jamal♦

      30.1k11114225




      30.1k11114225









      asked Jun 28 at 1:58









      Ciasto piekarz

      386720




      386720




















          2 Answers
          2






          active

          oldest

          votes

















          up vote
          1
          down vote



          accepted










          Instead of joining the data of two tables in Python, you should une an SQL join query to ask the database do the work for you.



          You also should not inject parameters directly into the query and let the driver do it for you by means of parametrized queries.



          Lastly, a list-comprehension is better suited to build your output than using append.



          Proposed improvements:



          def get_logs_messages(request_id, destination=None):
          parameters = (request_id,)
          query = (
          'SELECT logs.modified_dt, logs.message '
          'FROM scripts_deployment_logs AS logs '
          'LEFT JOIN deployment ON logs.deployment_id = deployment.deployment_id '
          'WHERE deployment.request_id = %s'
          )

          if destination is not None:
          query += ' AND deployment.destination = %s'
          parameters = (request_id, destination)

          conn = MySQLdb.connect(**dbconfig)
          cursor = conn.cursor()
          cursor.execute(query, parameters)
          messages = [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]
          cursor.close()
          return messages





          share|improve this answer























          • yes list comprehension is certainly more performance intuitive.
            – Ciasto piekarz
            Jun 28 at 9:54

















          up vote
          1
          down vote













          In addition to Mathias excelent answer



          • You could use context managers, so the connection will close automagically, when you are done. See PEP#343


          • Don't string.format SQL queries, since this may lead to SQLi vulnerabilities.


          from contextlib import closing

          def get_logs_messages(request_id, destination=None):
          ...

          with closing(MySQLdb.connect(**dbconfig)) as conn:
          with conn as cursor:
          cursor.execute(query, parameters)
          return [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]





          share|improve this answer























          • Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
            – Mathias Ettinger
            Jun 28 at 19:50










          • I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
            – Ciasto piekarz
            Jun 29 at 2:47






          • 1




            @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
            – Mathias Ettinger
            Jun 29 at 10:14










          Your Answer




          StackExchange.ifUsing("editor", function ()
          return StackExchange.using("mathjaxEditing", function ()
          StackExchange.MarkdownEditor.creationCallbacks.add(function (editor, postfix)
          StackExchange.mathjaxEditing.prepareWmdForMathJax(editor, postfix, [["\$", "\$"]]);
          );
          );
          , "mathjax-editing");

          StackExchange.ifUsing("editor", function ()
          StackExchange.using("externalEditor", function ()
          StackExchange.using("snippets", function ()
          StackExchange.snippets.init();
          );
          );
          , "code-snippets");

          StackExchange.ready(function()
          var channelOptions =
          tags: "".split(" "),
          id: "196"
          ;
          initTagRenderer("".split(" "), "".split(" "), channelOptions);

          StackExchange.using("externalEditor", function()
          // Have to fire editor after snippets, if snippets enabled
          if (StackExchange.settings.snippets.snippetsEnabled)
          StackExchange.using("snippets", function()
          createEditor();
          );

          else
          createEditor();

          );

          function createEditor()
          StackExchange.prepareEditor(
          heartbeatType: 'answer',
          convertImagesToLinks: false,
          noModals: false,
          showLowRepImageUploadWarning: true,
          reputationToPostImages: null,
          bindNavPrevention: true,
          postfix: "",
          onDemand: true,
          discardSelector: ".discard-answer"
          ,immediatelyShowMarkdownHelp:true
          );



          );








           

          draft saved


          draft discarded


















          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f197392%2fpulling-log-messages-from-a-mysql-table%23new-answer', 'question_page');

          );

          Post as a guest






























          2 Answers
          2






          active

          oldest

          votes








          2 Answers
          2






          active

          oldest

          votes









          active

          oldest

          votes






          active

          oldest

          votes








          up vote
          1
          down vote



          accepted










          Instead of joining the data of two tables in Python, you should une an SQL join query to ask the database do the work for you.



          You also should not inject parameters directly into the query and let the driver do it for you by means of parametrized queries.



          Lastly, a list-comprehension is better suited to build your output than using append.



          Proposed improvements:



          def get_logs_messages(request_id, destination=None):
          parameters = (request_id,)
          query = (
          'SELECT logs.modified_dt, logs.message '
          'FROM scripts_deployment_logs AS logs '
          'LEFT JOIN deployment ON logs.deployment_id = deployment.deployment_id '
          'WHERE deployment.request_id = %s'
          )

          if destination is not None:
          query += ' AND deployment.destination = %s'
          parameters = (request_id, destination)

          conn = MySQLdb.connect(**dbconfig)
          cursor = conn.cursor()
          cursor.execute(query, parameters)
          messages = [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]
          cursor.close()
          return messages





          share|improve this answer























          • yes list comprehension is certainly more performance intuitive.
            – Ciasto piekarz
            Jun 28 at 9:54














          up vote
          1
          down vote



          accepted










          Instead of joining the data of two tables in Python, you should une an SQL join query to ask the database do the work for you.



          You also should not inject parameters directly into the query and let the driver do it for you by means of parametrized queries.



          Lastly, a list-comprehension is better suited to build your output than using append.



          Proposed improvements:



          def get_logs_messages(request_id, destination=None):
          parameters = (request_id,)
          query = (
          'SELECT logs.modified_dt, logs.message '
          'FROM scripts_deployment_logs AS logs '
          'LEFT JOIN deployment ON logs.deployment_id = deployment.deployment_id '
          'WHERE deployment.request_id = %s'
          )

          if destination is not None:
          query += ' AND deployment.destination = %s'
          parameters = (request_id, destination)

          conn = MySQLdb.connect(**dbconfig)
          cursor = conn.cursor()
          cursor.execute(query, parameters)
          messages = [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]
          cursor.close()
          return messages





          share|improve this answer























          • yes list comprehension is certainly more performance intuitive.
            – Ciasto piekarz
            Jun 28 at 9:54












          up vote
          1
          down vote



          accepted







          up vote
          1
          down vote



          accepted






          Instead of joining the data of two tables in Python, you should une an SQL join query to ask the database do the work for you.



          You also should not inject parameters directly into the query and let the driver do it for you by means of parametrized queries.



          Lastly, a list-comprehension is better suited to build your output than using append.



          Proposed improvements:



          def get_logs_messages(request_id, destination=None):
          parameters = (request_id,)
          query = (
          'SELECT logs.modified_dt, logs.message '
          'FROM scripts_deployment_logs AS logs '
          'LEFT JOIN deployment ON logs.deployment_id = deployment.deployment_id '
          'WHERE deployment.request_id = %s'
          )

          if destination is not None:
          query += ' AND deployment.destination = %s'
          parameters = (request_id, destination)

          conn = MySQLdb.connect(**dbconfig)
          cursor = conn.cursor()
          cursor.execute(query, parameters)
          messages = [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]
          cursor.close()
          return messages





          share|improve this answer















          Instead of joining the data of two tables in Python, you should une an SQL join query to ask the database do the work for you.



          You also should not inject parameters directly into the query and let the driver do it for you by means of parametrized queries.



          Lastly, a list-comprehension is better suited to build your output than using append.



          Proposed improvements:



          def get_logs_messages(request_id, destination=None):
          parameters = (request_id,)
          query = (
          'SELECT logs.modified_dt, logs.message '
          'FROM scripts_deployment_logs AS logs '
          'LEFT JOIN deployment ON logs.deployment_id = deployment.deployment_id '
          'WHERE deployment.request_id = %s'
          )

          if destination is not None:
          query += ' AND deployment.destination = %s'
          parameters = (request_id, destination)

          conn = MySQLdb.connect(**dbconfig)
          cursor = conn.cursor()
          cursor.execute(query, parameters)
          messages = [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]
          cursor.close()
          return messages






          share|improve this answer















          share|improve this answer



          share|improve this answer








          edited Jun 28 at 19:52


























          answered Jun 28 at 9:49









          Mathias Ettinger

          21.7k32875




          21.7k32875











          • yes list comprehension is certainly more performance intuitive.
            – Ciasto piekarz
            Jun 28 at 9:54
















          • yes list comprehension is certainly more performance intuitive.
            – Ciasto piekarz
            Jun 28 at 9:54















          yes list comprehension is certainly more performance intuitive.
          – Ciasto piekarz
          Jun 28 at 9:54




          yes list comprehension is certainly more performance intuitive.
          – Ciasto piekarz
          Jun 28 at 9:54












          up vote
          1
          down vote













          In addition to Mathias excelent answer



          • You could use context managers, so the connection will close automagically, when you are done. See PEP#343


          • Don't string.format SQL queries, since this may lead to SQLi vulnerabilities.


          from contextlib import closing

          def get_logs_messages(request_id, destination=None):
          ...

          with closing(MySQLdb.connect(**dbconfig)) as conn:
          with conn as cursor:
          cursor.execute(query, parameters)
          return [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]





          share|improve this answer























          • Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
            – Mathias Ettinger
            Jun 28 at 19:50










          • I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
            – Ciasto piekarz
            Jun 29 at 2:47






          • 1




            @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
            – Mathias Ettinger
            Jun 29 at 10:14














          up vote
          1
          down vote













          In addition to Mathias excelent answer



          • You could use context managers, so the connection will close automagically, when you are done. See PEP#343


          • Don't string.format SQL queries, since this may lead to SQLi vulnerabilities.


          from contextlib import closing

          def get_logs_messages(request_id, destination=None):
          ...

          with closing(MySQLdb.connect(**dbconfig)) as conn:
          with conn as cursor:
          cursor.execute(query, parameters)
          return [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]





          share|improve this answer























          • Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
            – Mathias Ettinger
            Jun 28 at 19:50










          • I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
            – Ciasto piekarz
            Jun 29 at 2:47






          • 1




            @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
            – Mathias Ettinger
            Jun 29 at 10:14












          up vote
          1
          down vote










          up vote
          1
          down vote









          In addition to Mathias excelent answer



          • You could use context managers, so the connection will close automagically, when you are done. See PEP#343


          • Don't string.format SQL queries, since this may lead to SQLi vulnerabilities.


          from contextlib import closing

          def get_logs_messages(request_id, destination=None):
          ...

          with closing(MySQLdb.connect(**dbconfig)) as conn:
          with conn as cursor:
          cursor.execute(query, parameters)
          return [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]





          share|improve this answer















          In addition to Mathias excelent answer



          • You could use context managers, so the connection will close automagically, when you are done. See PEP#343


          • Don't string.format SQL queries, since this may lead to SQLi vulnerabilities.


          from contextlib import closing

          def get_logs_messages(request_id, destination=None):
          ...

          with closing(MySQLdb.connect(**dbconfig)) as conn:
          with conn as cursor:
          cursor.execute(query, parameters)
          return [
          ': '.format(dt, message)
          for dt, message in cursor.fetchall()
          ]






          share|improve this answer















          share|improve this answer



          share|improve this answer








          edited Jun 28 at 21:28


























          answered Jun 28 at 11:04









          Ludisposed

          5,68621656




          5,68621656











          • Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
            – Mathias Ettinger
            Jun 28 at 19:50










          • I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
            – Ciasto piekarz
            Jun 29 at 2:47






          • 1




            @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
            – Mathias Ettinger
            Jun 29 at 10:14
















          • Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
            – Mathias Ettinger
            Jun 28 at 19:50










          • I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
            – Ciasto piekarz
            Jun 29 at 2:47






          • 1




            @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
            – Mathias Ettinger
            Jun 29 at 10:14















          Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
          – Mathias Ettinger
          Jun 28 at 19:50




          Since this is not part of the DB API 2.0, I wasn't sure if MySQLdb implemented such behaviours. Seems like every major DB driver adopted the practice anyways. And now that you have context managers, you can directly return the list-comprehension.
          – Mathias Ettinger
          Jun 28 at 19:50












          I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
          – Ciasto piekarz
          Jun 29 at 2:47




          I am using mysql.connector so I am doing from mysql.connector import MySQLConnection. on using @Ludisposed suggestion I get AttributeError: __exit__ after with conn as cursor:
          – Ciasto piekarz
          Jun 29 at 2:47




          1




          1




          @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
          – Mathias Ettinger
          Jun 29 at 10:14




          @Ciastopiekarz You can then change with conn as cursor: to with closing(conn.cursor()) as cursor:.
          – Mathias Ettinger
          Jun 29 at 10:14












           

          draft saved


          draft discarded


























           


          draft saved


          draft discarded














          StackExchange.ready(
          function ()
          StackExchange.openid.initPostLogin('.new-post-login', 'https%3a%2f%2fcodereview.stackexchange.com%2fquestions%2f197392%2fpulling-log-messages-from-a-mysql-table%23new-answer', 'question_page');

          );

          Post as a guest













































































          Popular posts from this blog

          Python Lists

          Aion

          JavaScript Array Iteration Methods