live555 rtsp服务器实战之doGetNextFrame

live555关于RTSP协议交互流程

live555的核心数据结构值之闭环双向链表

live555 rtsp服务器实战之createNewStreamSource

live555 rtsp服务器实战之doGetNextFrame

live555搭建实时播放rtsp服务器 

注意：该篇文章可能有些绕，最好跟着文章追踪下源码，不了解源码可能就是天书；

概要

        live555用于实际项目开发时，createNewStreamSource和doGetNextFrame是必须要实现的两个虚函数，一般会创建两个类来实现这两个函数：假如这两个类为H264LiveVideoServerMediaSubssion和H264FramedLiveSource；H264LiveVideoServerMediaSubssion为实时视频会话类，用于实现createNewStreamSource虚函数；

        H264FramedLiveSource为实时视频帧资源类，用户实现doGetNextFrame函数；

        那么这两个函数是什么时候被调用以及他们的作用是什么呢？本节将详细介绍；

声明：该文章基于H264视频源为基础分析，其他源类似；

        由于这两个类主要是自定义虚函数，所以存在一定的继承关系，首先需要明确这两个类的继承关系(本章只介绍H264FramedLiveSource)：

        H264FramedLiveSource:FramedSource:MediaSource:Medium

doGetNextFrame调用流程

        doGetNextFrame函数声明于FrameSource类：

virtual void doGetNextFrame() = 0;

        该声明为纯虚函数，所以必须有子类对该函数进行实现，H264FramedLiveSource类就是用于实现该函数；doGetNextFrame函数只有一个功能：获取视频帧；

void H264FramedLiveSource::doGetNextFrame()
{uint8_t *frame = new uint8_t[1024*1024];int len = 0;//获取一帧视频帧get_frame(&frame, len);//将帧长度赋值给父类的成员变量fFrameSizefFrameSize = len;//将帧数据赋值给父类的成员变量fTomemcpy(fTo, frame, len);delete [] frame;// nextTask() = envir().taskScheduler().scheduleDelayedTask(0,(TaskFunc*)FramedSource::afterGetting, this);//表示延迟0秒后再执行 afterGetting 函数afterGetting(this);return;
}

        fFrameSize和fTo的两个父类变量在流传输时会用到；那么问题来了：doGetNextFrame函数在整个流程中在哪里被调用？什么时候调用？

        首先doGetNextFrame函数是在PLAY信令交互的时候被调用；调用流程为：

handleCmd_PLAY->startStream->startPlaying(OnDemandServerMediaSubsession)->startPlaying(MediaSink)->continuePlaying(虚函数，子类H264or5VideoRTPSink实现)->continuePlaying(虚函数，子类MultiFramedRTPSink实现)->buildAndSendPacket->packFrame(MultiFramedRTPSink)->getNextFrame->doGetNextFrame

        下面详解介绍下这个流程，看过我的另一篇文章："live555 rtsp服务器实战之createNewStreamSource" 的都知道handleRequestBytes函数调用了handleCmd_SETUP；同样handleCmd_PLAY函数也是在这里被调用的；handleRequestBytes函数就是用来处理各种客户端信令交互信息的；handleRequestBytes函数如下：

void RTSPServer::RTSPClientConnection::handleRequestBytes(int newBytesRead)
{...if (urlIsRTSPS != fOurRTSPServer.fOurConnectionsUseTLS){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_redirect()\n");
#endifhandleCmd_redirect(urlSuffix);}else if (strcmp(cmdName, "OPTIONS") == 0){// If the "OPTIONS" command included a "Session:" id for a session that doesn't exist,// then treat this as an error:if (requestIncludedSessionId && clientSession == NULL){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 1)\n");
#endifhandleCmd_sessionNotFound();}else{// Normal case:handleCmd_OPTIONS();}}else if (urlPreSuffix[0] == '\0' && urlSuffix[0] == '*' && urlSuffix[1] == '\0'){// The special "*" URL means: an operation on the entire server.  This works only for GET_PARAMETER and SET_PARAMETER:if (strcmp(cmdName, "GET_PARAMETER") == 0){handleCmd_GET_PARAMETER((char const *)fRequestBuffer);}else if (strcmp(cmdName, "SET_PARAMETER") == 0){handleCmd_SET_PARAMETER((char const *)fRequestBuffer);}else{handleCmd_notSupported();}}else if (strcmp(cmdName, "DESCRIBE") == 0){handleCmd_DESCRIBE(urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);}else if (strcmp(cmdName, "SETUP") == 0){Boolean areAuthenticated = True;if (!requestIncludedSessionId){// No session id was present in the request.// So create a new "RTSPClientSession" object for this request.// But first, make sure that we're authenticated to perform this command:char urlTotalSuffix[2 * RTSP_PARAM_STRING_MAX];// enough space for urlPreSuffix/urlSuffix'\0'urlTotalSuffix[0] = '\0';if (urlPreSuffix[0] != '\0'){strcat(urlTotalSuffix, urlPreSuffix);strcat(urlTotalSuffix, "/");}strcat(urlTotalSuffix, urlSuffix);if (authenticationOK("SETUP", urlTotalSuffix, (char const *)fRequestBuffer)){clientSession = (RTSPServer::RTSPClientSession *)fOurRTSPServer.createNewClientSessionWithId();}else{areAuthenticated = False;}}if (clientSession != NULL){clientSession->handleCmd_SETUP(this, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);playAfterSetup = clientSession->fStreamAfterSETUP;}else if (areAuthenticated){
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 2)\n");
#endifhandleCmd_sessionNotFound();}}else if (strcmp(cmdName, "TEARDOWN") == 0 || strcmp(cmdName, "PLAY") == 0 || strcmp(cmdName, "PAUSE") == 0 || strcmp(cmdName, "GET_PARAMETER") == 0 || strcmp(cmdName, "SET_PARAMETER") == 0){if (clientSession != NULL){clientSession->handleCmd_withinSession(this, cmdName, urlPreSuffix, urlSuffix, (char const *)fRequestBuffer);}else{
#ifdef DEBUGfprintf(stderr, "Calling handleCmd_sessionNotFound() (case 3)\n");
#endifhandleCmd_sessionNotFound();}}else if (strcmp(cmdName, "REGISTER") == 0 || strcmp(cmdName, "DEREGISTER") == 0){// Because - unlike other commands - an implementation of this command needs// the entire URL, we re-parse the command to get it:char *url = strDupSize((char *)fRequestBuffer);if (sscanf((char *)fRequestBuffer, "%*s %s", url) == 1){// Check for special command-specific parameters in a "Transport:" header:Boolean reuseConnection, deliverViaTCP;char *proxyURLSuffix;parseTransportHeaderForREGISTER((const char *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);handleCmd_REGISTER(cmdName, url, urlSuffix, (char const *)fRequestBuffer, reuseConnection, deliverViaTCP, proxyURLSuffix);delete[] proxyURLSuffix;}else{handleCmd_bad();}delete[] url;}else{// The command is one that we don't handle:handleCmd_notSupported();}...
}

        handleCmd_withinSession函数内部就调用了handleCmd_PLAY函数；仅整理流程，调用途中的函数这里不做解释；直接跳到packFrame(MultiFramedRTPSink)函数：

void MultiFramedRTPSink::packFrame()
{// Get the next frame.// First, skip over the space we'll use for any frame-specific header:fCurFrameSpecificHeaderPosition = fOutBuf->curPacketSize();fCurFrameSpecificHeaderSize = frameSpecificHeaderSize();fOutBuf->skipBytes(fCurFrameSpecificHeaderSize);fTotalFrameSpecificHeaderSizes += fCurFrameSpecificHeaderSize;// See if we have an overflow frame that was too big for the last pktif (fOutBuf->haveOverflowData()){// Use this frame before reading a new one from the sourceunsigned frameSize = fOutBuf->overflowDataSize();struct timeval presentationTime = fOutBuf->overflowPresentationTime();unsigned durationInMicroseconds = fOutBuf->overflowDurationInMicroseconds();fOutBuf->useOverflowData();afterGettingFrame1(frameSize, 0, presentationTime, durationInMicroseconds);}else{// Normal case: we need to read a new frame from the sourceif (fSource == NULL)return;fSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),afterGettingFrame, this, ourHandleClosure, this);}
}

        这里调用了getNextFrame函数，来看下getNextFrame函数的实现：

void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,afterGettingFunc* afterGettingFunc,void* afterGettingClientData,onCloseFunc* onCloseFunc,void* onCloseClientData) {// Make sure we're not already being read:if (fIsCurrentlyAwaitingData) {envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";envir().internalError();}fTo = to;fMaxSize = maxSize;fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()fAfterGettingFunc = afterGettingFunc;fAfterGettingClientData = afterGettingClientData;fOnCloseFunc = onCloseFunc;fOnCloseClientData = onCloseClientData;fIsCurrentlyAwaitingData = True;doGetNextFrame();
}

        发现啦！发现doGetNextFrame函数啦！所以doGetNextFrame函数就是在getNextFrame内被调用的；但是问题来了：搜索可以发现在live555中doGetNextFrame函数很多；怎么就确定这里的doGetNextFrame和我们自定义的doGetNextFrame有关呢？

        问的好！那这的关键点就在于fSource->getNextFrame的fSource变量到底是什么类的对象；才能确定doGetNextFrame到底调用的是哪个类的函数；fSource属于MediaSink类的成员变量:

FramedSource* fSource;

        但是FrameSource子类很多；所以需要确定fSource到底指向的是哪个子类；

        既然fSource是MediaSink类的成员，那么fSource肯定是在MediaSink或其子类中被赋值；因此查找下这些类；首先明确下MediaSink的继承关系,从fSource调用的类开始查找：

//父类
MultiFramedRTPSink:RTPSink:MediaSink:Medium
//子类
H264VideoRTPSink:H264or5VideoRTPSink:VideoRTPSink:MultiFramedRTPSink

        因此在这些类中查找即可；根据调用流程可知startPlaying最先赋值，值是startPlaying的第一个参数；

Boolean MediaSink::startPlaying(MediaSource& source,afterPlayingFunc* afterFunc,void* afterClientData) {// Make sure we're not already being played:if (fSource != NULL) {envir().setResultMsg("This sink is already being played");return False;}// Make sure our source is compatible:if (!sourceIsCompatibleWithUs(source)) {envir().setResultMsg("MediaSink::startPlaying(): source is not compatible!");return False;}fSource = (FramedSource*)&source;fAfterFunc = afterFunc;fAfterClientData = afterClientData;return continuePlaying();
}

        而该函数是在StreamState类中的startPlaying函数中调用；

void StreamState ::startPlaying(Destinations *dests, unsigned clientSessionId,TaskFunc *rtcpRRHandler, void *rtcpRRHandlerClientData,ServerRequestAlternativeByteHandler *serverRequestAlternativeByteHandler,void *serverRequestAlternativeByteHandlerClientData)
{...if (!fAreCurrentlyPlaying && fMediaSource != NULL){if (fRTPSink != NULL){fRTPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);fAreCurrentlyPlaying = True;}else if (fUDPSink != NULL){fUDPSink->startPlaying(*fMediaSource, afterPlayingStreamState, this);fAreCurrentlyPlaying = True;}}
}

        startPlaying的第一个参数是fMediaSource；而fMediaSource是在StreamState类的构造函数中被赋值的：

StreamState::StreamState(OnDemandServerMediaSubsession &master,Port const &serverRTPPort, Port const &serverRTCPPort,RTPSink *rtpSink, BasicUDPSink *udpSink,unsigned totalBW, FramedSource *mediaSource,Groupsock *rtpGS, Groupsock *rtcpGS): fMaster(master), fAreCurrentlyPlaying(False), fReferenceCount(1),fServerRTPPort(serverRTPPort), fServerRTCPPort(serverRTCPPort),fRTPSink(rtpSink), fUDPSink(udpSink), fStreamDuration(master.duration()),fTotalBW(totalBW), fRTCPInstance(NULL) /* created later */,fMediaSource(mediaSource), fStartNPT(0.0), fRTPgs(rtpGS), fRTCPgs(rtcpGS)
{
}

        StreamState是在OnDemandServerMediaSubsession类的getStreamParameters函数中被调用：

void OnDemandServerMediaSubsession ::getStreamParameters(unsigned clientSessionId,struct sockaddr_storage const &clientAddress,Port const &clientRTPPort,Port const &clientRTCPPort,int tcpSocketNum,unsigned char rtpChannelId,unsigned char rtcpChannelId,TLSState *tlsState,struct sockaddr_storage &destinationAddress,u_int8_t & /*destinationTTL*/,Boolean &isMulticast,Port &serverRTPPort,Port &serverRTCPPort,void *&streamToken)
{if (addressIsNull(destinationAddress)){// normal case - use the client address as the destination address:destinationAddress = clientAddress;}isMulticast = False;if (fLastStreamToken != NULL && fReuseFirstSource){// Special case: Rather than creating a new 'StreamState',// we reuse the one that we've already created:serverRTPPort = ((StreamState *)fLastStreamToken)->serverRTPPort();serverRTCPPort = ((StreamState *)fLastStreamToken)->serverRTCPPort();++((StreamState *)fLastStreamToken)->referenceCount();streamToken = fLastStreamToken;}else{// Normal case: Create a new media source:unsigned streamBitrate;FramedSource *mediaSource = createNewStreamSource(clientSessionId, streamBitrate);...streamToken = fLastStreamToken = new StreamState(*this, serverRTPPort, serverRTCPPort, rtpSink, udpSink,streamBitrate, mediaSource,rtpGroupsock, rtcpGroupsock);}
}

        createNewStreamSource函数加上OnDemandServerMediaSubsession类是不是很熟悉？对的OnDemandServerMediaSubsession就是我们自定义的用于实现createNewStreamSource函数的类H264LiveVideoServerMediaSubssion的父类；因为createNewStreamSource在OnDemandServerMediaSubsession中是纯虚函数，因此该处调用的就是我们自定义的createNewStreamSource函数；

        因此在startPlaying函数中将fSource赋值为createNewStreamSource的返回值；我们再看一下createNewStreamSource函数吧：

FramedSource* H264LiveVideoServerMediaSubssion::createNewStreamSource(unsigned clientSessionId, unsigned& estBitrate)
{/* Remain to do : assign estBitrate */estBitrate = 1000; // kbps, estimate//创建视频源H264FramedLiveSource* liveSource = H264FramedLiveSource::createNew(envir(), Server_datasize, Server_databuf, Server_dosent);if (liveSource == NULL){return NULL;}// Create a framer for the Video Elementary Stream:return H264VideoStreamFramer::createNew(envir(), liveSource);
}

        则fSource就是H264VideoStreamFramer::createNew(envir(), liveSource); 因此getNextFrame函数中运行的doGetNextFrame就是H264VideoStreamFramer中的doGetNextFrame函数；

        等等！发现问题没有：这个并不是我们自定义的H264FramedLiveSource类中获取视频帧的函数doGetNextFrame；这是为什么呢？

        别急！返回值中H264VideoStreamFramer::createNew(envir(), liveSource); 将doGetNextFrame的类对象liveSource传递进了H264VideoStreamFramer类中；而liveSource最终赋值给了StreamParser类中成员变量fInputSource和FramedFilter类的成员变量fInputSource；更多细节参考我的另一篇文章；

        接着往下说：H264VideoStreamFramer中的doGetNextFrame函数会调用MPEGVideoStreamFramer中的doGetNextFrame函数；在这个函数中有一个ensureValidBytes1第一次调用了自定义的函数doGetNextFrame

void StreamParser::ensureValidBytes1(unsigned numBytesNeeded) {
.
.
.fInputSource->getNextFrame(&curBank()[fTotNumValidBytes],maxNumBytesToRead,afterGettingBytes, this,onInputClosure, this);throw NO_MORE_BUFFERED_INPUT;
}

这里仅仅是对流的解析；下面才是真正获取视频流：

void H264or5Fragmenter::afterGettingFrame(void* clientData, unsigned frameSize,unsigned numTruncatedBytes,struct timeval presentationTime,unsigned durationInMicroseconds) {H264or5Fragmenter* fragmenter = (H264or5Fragmenter*)clientData;fragmenter->afterGettingFrame1(frameSize, numTruncatedBytes, presentationTime,durationInMicroseconds);
}void H264or5Fragmenter::afterGettingFrame1(unsigned frameSize,unsigned numTruncatedBytes,struct timeval presentationTime,unsigned durationInMicroseconds) {fNumValidDataBytes += frameSize;fSaveNumTruncatedBytes = numTruncatedBytes;fPresentationTime = presentationTime;fDurationInMicroseconds = durationInMicroseconds;// Deliver data to the client:doGetNextFrame();
}void H264or5Fragmenter::doGetNextFrame() {if (fNumValidDataBytes == 1) {// We have no NAL unit data currently in the buffer.  Read a new one:fInputSource->getNextFrame(&fInputBuffer[1], fInputBufferSize - 1,afterGettingFrame, this,FramedSource::handleClosure, this);} else {...}
}

        afterGettingFrame函数才是真正调用doGetNextFrame的函数？那么afterGettingFrame是在哪里被调用的呢？

        看一下fSource->getNextFrame函数：

fInputSource->getNextFrame(fOutBuf->curPtr(), fOutBuf->totalBytesAvailable(),afterGettingFrame, this, ourHandleClosure, this);

        这里第三个参数就调用了afterGettingFrame函数，再看看getNextFrame函数实现；上面已经写过了 为了方便理解这里再写下：

void FramedSource::getNextFrame(unsigned char* to, unsigned maxSize,afterGettingFunc* afterGettingFunc,void* afterGettingClientData,onCloseFunc* onCloseFunc,void* onCloseClientData) {// Make sure we're not already being read:if (fIsCurrentlyAwaitingData) {envir() << "FramedSource[" << this << "]::getNextFrame(): attempting to read more than once at the same time!\n";envir().internalError();}fTo = to;fMaxSize = maxSize;fNumTruncatedBytes = 0; // by default; could be changed by doGetNextFrame()fDurationInMicroseconds = 0; // by default; could be changed by doGetNextFrame()fAfterGettingFunc = afterGettingFunc;fAfterGettingClientData = afterGettingClientData;fOnCloseFunc = onCloseFunc;fOnCloseClientData = onCloseClientData;fIsCurrentlyAwaitingData = True;doGetNextFrame();
}

        afterGettingFrame被赋值给了fAfterGettingClientData指针；那么fAfterGettingClientData指针什么时候被调用的？

        回望doGetNextFrame的实现，每次读完流都会调用:

afterGetting(this);
//函数实现
void FramedSource::afterGetting(FramedSource* source) {source->nextTask() = NULL;source->fIsCurrentlyAwaitingData = False;// indicates that we can be read again// Note that this needs to be done here, in case the "fAfterFunc"// called below tries to read another frame (which it usually will)if (source->fAfterGettingFunc != NULL) {(*(source->fAfterGettingFunc))(source->fAfterGettingClientData,source->fFrameSize, source->fNumTruncatedBytes,source->fPresentationTime,source->fDurationInMicroseconds);}
}

        afterGetting的函数实现中就调用了fAfterGettingFunc指针；这就使doGetNextFrame形成了闭环：

        

至此函数doGetNextFrame的执行流程已经全部解析完毕，后续还会继续更新关于doGetNextFrame函数获取的帧数据是怎么处理和发送的，H264VideoStreamFramer中的doGetNextFrame函数和MPEGVideoStreamFramer中的doGetNextFrame函数都是什么作用！期待的话关注我，了解最新动态！

该文章持续更新！如果有错误或者模糊的地方欢迎留言探讨！