现在的位置: 首页 > 综合 > 正文

Linux下通过php对Mysql中的数据进行定时统计并入库

2013年01月10日 ⁄ 综合 ⁄ 共 4082字 ⁄ 字号 评论关闭

本文讲解Linux下对数据库的数据进行统计计算并将结果入库到另一个表中。本文分3部分进行讲解:第一部分介绍:linux下执行php的命令;第二部分介绍:通过脚本实现统计前几个月的数据并入库;第三部分介绍:如何实现每日凌晨对昨日的数据进行统计并入库。

1.Linux下执行php的命令

linux下执行php文件的命令为:php filename arguments

即:php 文件名 参数

Eg/usr/local/php/bin/php /home/zhaolincheung/insert_talkcountdata.php

2.通过脚本实现统计前几个月的数据并入库

例如,将之前60天内的数据统计入库。

首先,写一个shell文件talkcount_insert_db.sh,循环计算日期,并将该日期传给php文件,进行统计并入库。shell如下:

#!/bin/sh
for((i=1;i<=60;i++));
do
    echo -e $i":";
    day=`date -d "-"$i" day" +%Y-%m-%d`;
    /usr/local/php/bin/php /home/zhaolincheung/insert_talkcountdata.php $day;
    echo "";
done

而insert_talkcountdata.php是对某一天的数据统计并入库,如下:

<?php
  include_once('/data/analys/DB/MyDB.class.php');
  
    //定义大类的编号
    $cidArr = array(0,2,1,11,15,14,7,6,4,12,5,13,16,3,9,1001,1002);

    $yesterday = $_SERVER["argv"][1]; //获取php执行文件的参数
    echo $yesterday . "\n";

    //遍历大类并入库
    foreach($cidArr as $cid){
        //统计大类
        insert_db($cid,$yesterday); 
    }
    
    //根据指定的大类和日期,计算该天的消息量并入库
    function insert_db($category, $date){
        $dateArr = explode("-", $date);
        if(checkdate($dateArr[1], $dateArr[2], $dateArr[0]) == false){
            return false;
        }
        //将日期转为时间戳
        $thisdate = mktime(0, 0, 0, $dateArr[1], $dateArr[2], $dateArr[0]);
        $nextdate = $thisdate + 3600 * 24;

        $sql = "select talkid,count(*) as msgcount from table1 where postcategoryid=$category ";
        $sql .= " and updatetime >= $thisdate and updatetime < $nextdate group by talkid order by msgcount desc";

        $db = new MyDB();
        $res = $db->query($sql);
        
	    //初始化关联数组
        $mc["c20"]=$mc["c10"]=$mc["c6"]=$mc["c5"]=$mc["c4"]=$mc["c3"]=$mc["c2"]=$mc["c1"]=0;
        //分段统计消息数对应的会话数
        while(($result = $db->fetch_assoc($res))) 
        {
          $msgcount = $result["msgcount"];
          if($msgcount > 20)
            $mc["c20"]++;
          else if($msgcount > 10 && $msgcount <= 20)
            $mc["c10"]++;
          else if($msgcount >= 6 && $msgcount <= 10)
            $mc["c6"]++;
          else if($msgcount == 5)
            $mc["c5"]++;
          else if($msgcount == 4)
            $mc["c4"]++;
          else if($msgcount == 3)
            $mc["c3"]++;
          else if($msgcount == 2)
            $mc["c2"]++;
          else if($msgcount == 1)
            $mc["c1"]++;
        }
        
        //insert table 'talkcount'
        $sql_insert = "insert into talkcount(analysisdate,postcategoryid,c20,c10,c6,c5,c4,c3,c2,c1) values('";
        $sql_insert .= $date . "',";
        $sql_insert .= $category . ",";
        $sql_insert .= $mc["c20"] . ",";
        $sql_insert .= $mc["c10"] . ",";
        $sql_insert .= $mc["c6"] . ",";
        $sql_insert .= $mc["c5"] . ",";
        $sql_insert .= $mc["c4"] . ",";
        $sql_insert .= $mc["c3"] . ",";
        $sql_insert .= $mc["c2"] . ",";
        $sql_insert .= $mc["c1"] . ")";
        
	    echo $sql_insert . "\n";
        
        $db->query($sql_insert);
        
	    $db->close();
    }
?>

3.如何实现每日凌晨对昨日的数据进行统计并入库

另外,每日凌晨都需要将昨日的数据进行计算并入库。这里可以采用crontab -e命令,在计划任务中加入要执行的php文件。如:

0 5 * * * /usr/local/php/bin/php /data/analys/php/talkcount_Insertdb.php

表示每天凌晨五点执行/data/analys/php目录下的talkcount_Insertdb.php这个文件。

talkcount_Insertdb.php的内容如下:

<?php
  include_once('/data/analys/DB/MyDB.class.php');
  
    //定义大类的编号
    $cidArr = array(0,2,1,11,15,14,7,6,4,12,5,13,16,3,9,1001,1002);

    $yesterday = date("Y-m-d",strtotime("-1 day")); //计算昨天的日期
    echo $yesterday . "\n";

    //遍历大类并入库
    foreach($cidArr as $cid){
        //统计大类
        insert_db($cid,$yesterday); 
    }
    
    //根据指定的大类和日期,计算该天的消息量并入库
    function insert_db($category, $date){
        $dateArr = explode("-", $date);
        if(checkdate($dateArr[1], $dateArr[2], $dateArr[0]) == false){
            return false;
        }
        //将日期转为时间戳
        $thisdate = mktime(0, 0, 0, $dateArr[1], $dateArr[2], $dateArr[0]);
        $nextdate = $thisdate + 3600 * 24;

        $sql = "select talkid,count(*) as msgcount from table1 where postcategoryid=$category ";
        $sql .= " and updatetime >= $thisdate and updatetime < $nextdate group by talkid order by msgcount desc";

        $db = new MyDB();
        $res = $db->query($sql);
        
	    //初始化关联数组
        $mc["c20"]=$mc["c10"]=$mc["c6"]=$mc["c5"]=$mc["c4"]=$mc["c3"]=$mc["c2"]=$mc["c1"]=0;
        //分段统计消息数对应的会话数
        while(($result = $db->fetch_assoc($res))) 
        {
          $msgcount = $result["msgcount"];
          if($msgcount > 20)
            $mc["c20"]++;
          else if($msgcount > 10 && $msgcount <= 20)
            $mc["c10"]++;
          else if($msgcount >= 6 && $msgcount <= 10)
            $mc["c6"]++;
          else if($msgcount == 5)
            $mc["c5"]++;
          else if($msgcount == 4)
            $mc["c4"]++;
          else if($msgcount == 3)
            $mc["c3"]++;
          else if($msgcount == 2)
            $mc["c2"]++;
          else if($msgcount == 1)
            $mc["c1"]++;
        }
        
        //insert table 'talkcount'
        $sql_insert = "insert into talkcount(analysisdate,postcategoryid,c20,c10,c6,c5,c4,c3,c2,c1) values('";
        $sql_insert .= $date . "',";
        $sql_insert .= $category . ",";
        $sql_insert .= $mc["c20"] . ",";
        $sql_insert .= $mc["c10"] . ",";
        $sql_insert .= $mc["c6"] . ",";
        $sql_insert .= $mc["c5"] . ",";
        $sql_insert .= $mc["c4"] . ",";
        $sql_insert .= $mc["c3"] . ",";
        $sql_insert .= $mc["c2"] . ",";
        $sql_insert .= $mc["c1"] . ")";
        
	    echo $sql_insert . "\n";
        
        $db->query($sql_insert);
        
	    $db->close();
    }
?>

抱歉!评论已关闭.